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A  INTRODUCTION 

For  several  decades,  the  definitive  treatment  for  low  and  medium  risk  prostate  cancer  was  radical  prostatectomy 
or  external  beam  radiation  therapy,  but  low  dose  rate  permanent  seed  brachytherapy  (shortly  brachytherapy) 
today  can  achieve  equivalent  outcomes.  Brachytherapy,  if  accurately  executed,  can  achieve  a  sharp  demarcation 
between  the  treated  volume  and  healthy  structures,  and  thereby  achieve  superior  tumor  control  with  reduced 
morbidity.  In  contemporary  practice,  however,  faulty  needle  and  source  placement  often  cause  insufficient  dose 
to  the  cancer  and/or  inadvertent  radiation  of  the  rectum,  urethra,  and  bladder.  Another  fallacy  of  the  current 
implant  techniques  is  that  reliable  and  accurate  exit  dosimetry  is  not  possible.  The  contribution  of  the  proposed 
research  will  be  making  C-arm  fluoroscopy  available  for  safe,  simple,  and  robust  intra-operative  localization  of 
brachytherapy  sources  relative  to  the  prostate.  We  will  develop  a  method  for  the  registration  of  ultrasound  to 
fluoroscopy  (RUF),  to  fuse  TRUS  (Trans-rectal  ultrasound  can  view  the  prostate  but  not  the  seeds)  with  C-arm 
fluoroscopy  (which  is  capable  of  viewing  the  seeds  but  not  the  prostate).  This  feature  will  allow  for  dosimetric 
optimization  of  the  prostate  brachytherapy  implants  and  exit  dosimetry  before  the  patient  is  released  from  the 
operating  room;  thereby  enabling  significant  improvement  on  current  clinical  practice.  A  further  promise  is  that 
fluoroscopy-based  exit  dosimetry  may  obviate  CT-based  post-implant  dosimetry. 


Figure  1: 


Initial  plan  - 


r 

US  imaging 


insert  needle 


X-ray  imaging 


B  BODY 

B.l  Brief  System  Concept 

The  system  concept  for  registration  of  ultrasound  to 
fluoroscopy  (RUF)  is  summarized  in  Figure  1.  The 
fluoroscope  is  calibrated  and  corrected  from  image 
distortion  pre-operatively.  The  implant  procedure  starts  as 
usual:  TRUS  is  used  to  guide  each  individual  needle  and  a 
C-arm  placed  over  the  patient’s  abdomen.  The  C-arm  is 
tracked  with  an  X-ray  fiducial  system  called  FTRAC  that  is 

composed  of  optimally  selected  polynomial  space  curves.  Figure  l.WjjteMconq^jK^^Ldnd  C-arm  fusion 

The  fiducials  are  mounted  rigidly  to  the  TRUS  frame  in  the  ifealB  BCrf? _ ! 

field  of  view  in  a  known  calibrated  pose  relative  to  the  TRUS  thereby  providing  spatial  rffgf&tl&tfflft  between  the 
C-arm  and  TRUS.  Upon  implanting  a  batch  of  needles  ftypiraflya-row  of  needles!,  we  collect  a  set  of  TRUS 
and  C-arm  images.  The  locations  of  the  implanted  seeds  are  recovered  from  the  C-arm  fluoroscopy  images  with 
the  use  of  a  network  flow  based  method  called  MARSHAL  that  provides  seed  segmentation,  matching,  and 
reconstruction  method.  Then  the  cloud  of  seeds  is  superimposed  over  the  spatially  co-registered  TRUS  images. 
The  3D  dose  distribution  is  rapidly  calculated  from  the  union  of  the  already  and  yet  to  be  implanted  seeds.  The 
dose  distribution  is  analyzed  with  tools  currently  available  in  the  brachytherapy  system  used.  Then  the  implant 
plan  can  be  optimized  to  account  for  discrepancies  from  the  ideal  dose  distribution.  The  procedure  continues 
with  the  next  batch  of  needles  in  the  cycle  described  above.  After  the  last  needle,  a  complete  dosimetry  check  is 
performed,  which  provides  a  final  opportunity  to  patch  up  any  cold  spots  with  additional  seeds. 


B.2  Proposed  Statement  of  Work 

We  proposed  to  develop  a  method  for  the  registration  of  ultrasound  to  fluoroscopy  (RUF)  to  allow  for  intra¬ 
operative  dosimetry  in  prostate  brachytherapy  and  prototype  mathematical  algorithms  (Aim-1),  integrate  them 
with  an  existing  FDA  approved  prostate  brachytherapy  system  that  provides  dosimetry  analysis  (Aim-2),  and 
evaluate  the  system  experimentally  on  phantoms  and  pre-recorded  patient  data  (Aim-3).  Algorithmic  design 
(Aim-1)  and  experimental  evaluation  (Aim-3),  will  progress  hand  in  hand.  System  integration  (Aim-2)  will  be 
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performed  immediately  when  a  workable  subset  of  RUF  package  becomes  available  from  Aim- 1  and  again 
revisited  towards  at  the  end  of  the  project.  Therefore,  the  timeline  will  be  somewhat  non-linear.  The  detailed 
statement  of  work  was  as  follows: 

Aim-1:  Registration  of  Ultrasound  to  Fluoroscopy  (RUF):  Develop  a  methods  for  reconstruction  of  seed 
implants  from  X-ray  fluoroscopy  and  spatially  registering  them  to  the  prostate  anatomy  identified  in 
TRUS 

Aim-2:  System  Integration:  Integrate  the  above  methods  in  a  software  package  and  link  it  with  the  FDA- 
approved  CMS  Interplant®  prostate  brachytherapy  system  to  enable  dosimetry  calculation 
Aim-3:  Experimental  Validation:  Evaluate  the  performance  of  the  RUF  system  on  phantoms  and  pre¬ 
recorded  patient  data.  (Neither  of  which  require  an  IRB  approval) 

B.3  Progress  Report  for  Second  Year 

In  the  previous  report  in  Nov  2007,  we  accomplished  all  the  proposed  project  aims.  We  made  further  progress, 
in  addition  to  what  was  promised,  and  delivered  a  system  that  worked  intra-operatively  during  the  surgery  (vs  a 
system  that  will  successfully  be  validated  on  pre-recorded  data). 

This  extension  was  taken,  predominantly  to  document  all  the  work  into  publications  and  present  them.  The  goal 
was  successfully  met,  and  a  total  of  4  publications  were  presented  from  Oct  2007  -  March  2008,  one  was 
accepted  for  publication  in  May  2008,  while  another  one  submitted  for  a  workshop  in  June.  The  final  system 
paper  received  an  award  at  the  prestigious  SPIE  International  symposium  on  Medical  Imaging  2008. 

C  KEY  RESEARCH  ACCOMPLISHMENTS 

1 .  Completed  all  the  data  analysis  from  various  phantom  and  patient  trials. 

D  KEY  TRAINING  ACCOMPLISHMENTS 

1 .  Training  courses  in  medical  imaging  processing. 

E  REPORTABLE  OUTCOMES 

1.  Presented  4  publications,  as  listed  in  the  publications  section  (including  three  flagship  conference 
publications  in  MICCAI),  got  one  more  accepted,  and  submitted  another  one. 

2.  A  poster  award  at  SPIE  Medical  Imaging  2008  for  the  proposed  system  in  the  PhD  thesis. 

F  CONCLUSIONS 

In  conclusion,  the  system  was  exquisitely  accurate  in  clinical  trials  and  has  shown  usefulness.  We  require 
minimal  alteration  to  the  current  clinical  protocol  or  any  significant  increase  in  cost. 

So  what:  The  success  of  brachytherapy  chiefly  depends  on  our  ability  to  intra-operatively  cover  the  prostate 
with  sufficient  radiation  while  still  avoiding  excessive  radiation  to  surrounding  organs.  Currently,  such  level  of 
precision  is  not  always  achievable  even  by  the  most  experienced  physicians.  Thus  many  implants  fail  or  cause 
severe  side  effects  owing  to  faulty  seed  placement,  a  problem  what  still  cannot  be  corrected  in  the  operating 
room.  Our  results  indicate  the  feasibility  of  a  system  that  could  achieve  intra-operative  localization  of  the 
implanted  seeds  in  relation  to  the  prostate,  to  allow  for  in-situ  dosimetric  optimization  and  exit  dosimetry.  This 
ability  to  perform  intra-operative  dosimetry  may  change  the  standard  of  care  in  brachytherapy  by  allowing  the 
physician  to  achieve  technically  excellent  brachytherapy  implants,  resulting  in  improved  disease  control  and 
quality  of  life  for  a  large  and  steadily  growing  group  of  patients. 
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Abstract 

C-arm  fluoroscopy  is  ubiquitous  in  contemporary 
surgery t  but  it  lacks  the  ability  to  accurately  reconstruct 
three-dimensional  information,  attributable  to  the  difficulty 
in  obtaining  the  pose  of  X-ray  images  in  3D  space.  We 
propose  a  unified  mathematical  framework  to  address  the 
issues  of  intra-operative  pose  estimation,  correspondence 
and  reconstruction ,  using  simple  elliptic  curves.  In  contrast 
to  other  fiducial-based  tracking  methods,  our  method  uses  a 
single  ellipse  to  constrain  5  out  of  6  degrees  of  freedom  of  C- 
arm  pose ,  along  with  randomly  distributed  unknown  points 
in  the  imaging  volume  (either  naturally  present  or  induced 
by  randomly  placed  beads  or  other  markers  in  the  image 
space)  from  two  images/views  to  completely  recover  the  C- 
arm  pose.  Preliminary  phantom  experiments  indicate  an 
average  C-arm  tracking  accuracy  0/0,51°  and  0.12°  STD. 
The  method  appears  to  be  sufficiently  accurate  and  appeal - 
ing  for  many  clinical  applications,  since  it  uses  a  simple 
elliptic  fiducial  coupled  with  patient  information  and  has 
very  minimal  interference  with  the  workspace , 

1,  Introduction 

C-arm  fluoroscopy  is  the  most  commonly  used  intra¬ 
operative  imaging  modality  because  of  its  low  cost  and  case 
of  use,  but  it  currently  lacks  the  ability  for  robust  and  easy 
quantitative  guidance.  For  intra-operative  quantitative  anal¬ 
ysis,  one  needs  to  address  the  issues  of  1 )  image  distortion; 
2)  calibration  of  imaging  parameters;  3)  pose  estimation; 
and  4)  registration  to  other  imaging  modalities.  The  first 
two  problems  are  well  studied  in  literature  and  several  solu¬ 
tions  have  been  proposed  to  date  [  ,  ,  f  On  the  other 

hand,  pose  recovery  on  uncneuded  mobile  C-arms  is  still 
a  major  technical  problem  that  presently  docs  not  have  a 
practical  solution  in  many  areas  of  application.  Adding  to 
the  problem  is  the  non-isocentric  nature  of  C-arms,  where 
the  two  axes  of  rotation  does  not  intersect.  Also,  it  has  been 
observed  that  calibration  parameters,  for  both  distortion  and 
image,  change  from  pose  to  pose,  making  it  even  harder  to 
achieve  accurate  pose  recovery. 


To  date,  a  wide  variety  of  approaches  have  been  pro¬ 
posed  to  recover  C-arm  pose,  ranging  from  external  track¬ 
ing  devices  [  ,  ],  precise  calibration  fiducials  consisting 

of  beads,  lines,  conics  etc  in  known  geometry  [  ,  , 

],  to  image-based  pose  estimation  methods  [  ,  |  adapted 
from  the  computer  vision  literature.  Auxiliary  tracking 
devices  arc  expensive  and  need  an  extra  calibration  step, 
optical  systems  have  line  of  sight  problems  and  electro¬ 
magnetic  trackers  suffer  from  distortion  from  metallic  in¬ 
struments,  thus  adding  to  the  complexity  of  the  operating 
room. 

Radio -opaque  fiducials  placed  in  the  field  of  imaging 
have  been  demonstrated  to  be  capable  of  achieving  reason¬ 
able  accuracies.  However,  these  fiducials  need  to  be  man¬ 
ufactured  and  segmented  precisely  and  the  accuracies  de¬ 
pend  on  the  size  of  the  fiducial,  the  number  of  features  and 
the  type  of  features,  adding  to  the  segmentation  and  imag¬ 
ing  volume  problems.  It  has  been  shown  that  ellipses  serve 
as  ideal  features  as  a  3D  ellipse  would  project  as  a  closed 
ellipse  in  the  2D  images  [  ,  ].  A  circle  is  a  special  case  of 
ellipse  and  has  been  used  widely  lor  pose  estimation,  either 
alone  or  with  other  features  [  ,  ,  ]. 

In  image-based  methods,  point  correspondences  across 
the  images,  without  the  knowledge  about  their  3D  loca¬ 
tions,  offer  very  strong  constraints  for  determining  the  C- 
arm  pose,  popularly  known  as  bundle  adjustment  in  com¬ 
puter  vision  parlance |  ].  Jain  et  al.  proposed  a  mathemat¬ 
ical  framework  to  solve  for  pose  parameters,  point  corre¬ 
spondences,  and  reconstruction  using  a  high  dimensional 
non-linear  optimization  with  point  features  from  three  im¬ 
ages  [  ].  It  can  be  noticed  that  there  is  a  trade-off  between 
the  number  of  points  and  the  number  of  images  used  to 
determine  the  pose  accurately.  Determining  correct  point 
correspondences  depends  on  the  number  of  points,  spac¬ 
ing  between  the  points,  configuration  of  the  points  and  also 
on  the  number  of  images  to  eliminate  degenerate  matches. 
At  the  same  time,  increasing  the  number  of  images  adds 
to  the  complexity  of  the  non-linear  optimization  problem. 
While  some  procedures  may  be  more  tolerant  to  these  short¬ 
comings,  despite  pressing  clinical  needs,  better  approaches 
to  achieve  intra-operative  quantitative  fluoroscopy  that  fits 
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the  existing  clinical  workflow  arc  needed,  thereby  provid¬ 
ing  motivation  for  this  work. 

In  this  paper,  we  propose  a  unified  framework  to  cou¬ 
ple  fiducial-based  techniques  with  image-based  algorithms 
to  effectively  determine  C-arm  pose  from  two  or  more  im¬ 
ages.  We  are  using  a  single  ellipse  as  our  fiducial  because 
ellipses  serve  as  excellent  3D  marker  owing  to  their  con¬ 
strained  mathematical  framework  and  superior  image  seg¬ 
mentation,  as  reported  in  f  ],  We  present  a  detailed  math¬ 
ematical  framework  to  recover  pose  using  a  known  3D  el¬ 
lipse  and  its  2D  projective  images.  Our  mathematical  anal¬ 
ysis  shows  that  the  ellipse  from  a  single  image  offers  sym¬ 
metric  solutions  and  hence  can  only  constrain  5  out  of  6 
degrees  of  freedom.  To  recover  all  6  degrees  of  freedom, 
we  impose  more  constraints  in  the  form  of  point  correspon¬ 
dences  across  multiple  images.  It  can  be  argued  that  these 
extra  constraints  can  be  obtained  by  adding  more  features  to 
the  elliptic  fiducial  for  example  a  RB,  a  line,  or  even  a  sec¬ 
ond  ellipse.  Adding  more  features  increases  the  burden  of 
segmentation,  interference  with  the  anatomy  and  also  cre¬ 
ates  a  need  for  precise  manufacturing  and  extra  hardware 
in  the  operating  room.  Instead,  we  propose  to  use  multiple 
projections  and  feature/point  correspondences  across  these 
images  to  recover  the  pose,  thereby  using  patient  anatomical 
information  as  part  of  the  fiducial.  This  is  a  novel  method 
which  benefits  from  both  fiducial  based  methods  and  im¬ 
age  based  methods  as  the  fiducial  provides  a  good  initial 
estimate  for  the  point  correspondences  and  the  point  cor¬ 
respondences  in  turn  resolve  the  degenerate  cases  from  the 
single  ellipse. 

We  arc  targeting  two  pertinent  applications  for  this  work: 
1 )  brae  by  therapy  and  2)  orthopaedics.  Prostate  brae  hy  ther¬ 
apy  is  a  procedure  where  radio-active  seeds  arc  implanted 
into  the  prostate  to  treat  prostate  cancer.  C-arm  fluoroscopy 
is  used  to  get  real-time  dose  analysis  in  the  operating  room 
[  ],  Our  framework  fits  well  into  this  procedure  as  the 
ellipse  has  very  minimal  interference  with  the  anatomy 
and  at  the  same  time  we  can  also  use  the  radioactive  seeds 
as  our  point  features  to  accurately  determine  the  C-arm 
pose.  Moreover,  in  orthopaedic  applications,  it  is  very 
common  to  insert  screws,  surgical  markers  etc.,  that  can 
serve  as  our  point  fiducials.  In  applications  that  do  not  have 
many  point  features,  one  can  either  induce  some  markers 
or  identify  homologous  landmark  features  in  the  projection 
images  and  use  them  as  fiducials.  All  things  considered, 
we  believe  that  this  combined  approach  has  great  potential 
for  intra-operative  C-arm  tracking  that  would  fit  very  well 
into  existing  clinical  scenarios. 


2.  Method 

The  main  components  of  this  problem  arc:  1)  c-arm  pose 
and  2)  point  correspondences.  It  is  assumed  that  the  points 


have  been  segmented  from  the  X-ray  images.  C-arm  imag¬ 
ing  is  typically  approximated  as  a  5-paramctcr  pinhole  per¬ 
spective  projection  camera  [  ,  ,  ]  to  be  calibrated  inlra- 

operatively  for  each  individual  image.  Recently,  however,  it 
has  been  shown  that  precise  C-ann  calibration  does  not  sig¬ 
nificantly  improve  pose  reconstruction,  especially  the  rela¬ 
tive  pose  between  the  images  [  j.  We  assume  constant  cali¬ 
bration  across  the  images,  along  with  known  pixel  sizes.  We 
present  a  detailed  framework  to  estimate  C-arm  pose  using 
two  images,  though  this  framework  can  be  easily  extended 
to  three  or  more  images. 

2*L  Pose  Estimation  using  Single  Ellipse 

Given  a  known  3D  ellipse,  the  goal  here  is  to  estimate 
the  pose  of  this  ellipse  from  its  corresponding  projective 
images.  It  is  typical  to  represent  the  segmented  2D  ellipse 
as  an  equation  of  the  form: 

An2  +  Buv  4  Cu2  +  Du  4-  Fv  4-  G  =  0  ( 1 ) 

in  each  image,  in  which  u  and  v  arc  respectively  the  coor¬ 
dinates  along  the  x  and  //  axis  of  the  image,  measured  in 
pixels,  and  A ,  B ,  C,  D ,  F s  and  G  arc  constants  determined 
by  the  segmentation  algorithm.  It  can  be  observed  that  this 
correlates  with  the  general  equation  of  a  conic,  the  origin  of 
which  is  sitting  at  the  principal  point.  The  equation  of  such 
a  cone  will  appear  like 

A0x2  +  Boxy -I- Cox2  +  ^y-xz+  ~yz+  jjz2  =  0.  (2) 

where  xt  y ,  z  arc  coordinates  in  a  frame  that  has  the  focal 
point  of  the  x-ray  device  as  origin  with  z  axis  perpendicular 
to  the  image  plane  and  x  and  y  axis  parallel  to  the  imaging 
plane,  and  /  is  the  focal  length  of  the  imaging  scenario. 
This  equation  can  be  represented  in  matrix  form  as 
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(3) 


which  can  also  be  represented  as 

XM0Xr  =  0, 


(4) 


We  broke  the  process  into  several  steps  using  a  method  sim¬ 
ilar  to  that  described  for  the  case  of  a  circle  by  Costa  and 
Shapiro  [  1  and  the  detailed  derivation  can  be  found  in  [  ]. 
We  assume  for  simplicity  that  the  coordinate  frame  of  the 
3D  ellipse  has  the  x-axis  along  its  major  axis,  the  y-axis 
along  its  minor  axis,  and  the  origin  in  the  center  of  the  el¬ 
lipse.  The  first  step  involves  finding  the  rotation  between  the 
coordinate  frame  defined  by  the  image  plane  and  the  base 
coordinate  frame  of  the  cone  itself  (with  x  and  y  axes  along 
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Figure  1 .  Graphical  representation  of  the  angles  phi  and  theta,  used 
to  slice  the  cone  and  oh  tain  an  ellipse(left).  Algorithmic  flowchart 
showing  pose  estimation  with  point  correspondences  (right) 


the  major  and  minor  axis  of  the  cone  and  z  axis  along  the 
cone’s  center).  Note  that  the  rotation  matrix  that  achieves 
this  transformation  is  just  the  eigenvectors  of  the  original 
cone  matrix  Mq*  The  base  frame  of  the  cone  is  of  particular 
interest  because  in  it  the  cone  equation  has  special  form  of 
the  type 


'Aa  0  0  1 

X 

y  *  ] 

0  CB  0 

y 

0  0  7^ 

z 

(5) 


The  next  step  is  to  find  the  transformation  between  the 
coordinate  frame  of  the  base  cone  and  that  of  the  3D  ellipse. 
By  applying  the  constraints  imposed  by  the  physical  size  of 
the  ellipse,  we  attempt  to  evaluate  the  set  of  transformations 
that  would  achieve  this  for  a  given  image.  We  parameterize 
the  problem  by  defining  two  values,  and  8  which  together 
can  represent  a  rotation  from  the  base  frame  (z-axis  aligned 
with  the  cone)  to  an  arbitrary  frame.  To  model  this  transfor¬ 
mation,  the  base  frame  is  first  rotated  by  o  about  its  3  axis 
and  then  by  9  around  the  new  t/-axis.  This  transformation  is 
illustrated  in  Figure  J  a. 

For  any  image,  there  are  infinitely  many  possible  (0, 9) 
combinations,  corresponding  to  infinitely  many  physical  el¬ 
lipses  that  could  generate  the  observed  projected  ellipse  in 
the  X-ray  image.  These  rotations  can  be  expressed  by  the 
following  matrices: 


/?*- 


cos  tp  —  sin  4>  0 

sin  <f>  cos  <f>  0 

0  0  1 


Rb 


cos  0  0  sin  8 

0  1  0 
—  sin  9  0  cos  9 


(6) 

(7) 


Our  goal  therefore  is  to  solve  for  <p  and  9  such  that: 

X Rg  M p  Rg  /?.£ XT  =  0  (8) 

where  Mp  is  the  equation  for  the  cone  in  its  base  frame. 

Furthermore,  note  that  all  parallel  planes  slicing  the 
cone,  create  a  conic  curve  with  the  exact  same  eccentricity. 
In  our  case,  this  eccentricity  should  be  equal  to  the  eccen¬ 
tricity  of  the  3D  ellipse  (a  known  constant).  So  without  loss 
of  generality,  if  wc  set  Z  —  1,  we  will  get  the  equation  of 
an  ellipse,  such  as 

Ap  *  x2  Dp  *  xy  +  Cp  *  y2  —  K  (9) 

where 

Ap  —  Go  sin2  0  4- Co  cos2  8  sin2  <j>+ A#  cos2  0  cos2  (10) 

Bp  =  2{— Aq  cos  0  cos  sin  <p  +  Cq  cos  8  sin  <j>  cos  tfi} 

00 

Cp  —  Aq  sin2  <j>  +  Cq  cos2  (12) 

The  ellipse  generated  in  this  slice  can  also  described  by  a 
more  common  equation: 

(a;  cost/j  4-  y  sin^?)2  (x  sin  ip  —  y  cos^)2 

-  - - ^  _ — _ —  — — —  —  K  (13) 

a4  tr 

where  ip  is  a  rotation  around  the  z-axis  of  the  3D  ellipse,  a 
Sl  b  are  the  major  &  minor  axes  of  the  3D  ellipse.  K  will  be 
scale-constant  dependent  on  the  z  value  at  which  the  slice 
was  taken.  Expanding  this  equation  simplifies  to 


So,  from  equation  9,  wc  know  that  for  some  angle  of  rota¬ 
tion  ip: 


Combining  these  equations  to  eliminate  ip  and  K,  gives  us 

Af+cf.k(12  +  ^ 

(Ap  -  Cp)2  4-  Bp  _  (jpr  "  l)2 
(Ap  +  Cpf  (£  + 1)* 
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Note  that  if  we  were  imaging  a  circle,  the  right  hand  term 
reduces  to  0*  indicating  Bp  —  0,  and  Ap  —  Cp>  Substitut¬ 
ing  the  values  for  Ap,  Bp,  and  Cp  previously  presented  in 
equations  10,  II,  12  allows  us  to  obtain  an  equation  relating 
0  to  0.  This  equation  is  a  quadratic  equation  in  cos2(<j>)  of 
the  form,  with  the  Kn(0)  functions  of  0: 

ft'i(fl)cos4  0  +  K'2(0)  cos2  0  +  A  g{0j  =0  (15) 

Since  equation  15  is  a  bi-quadratic  equation  in  cos<j>,  for 
any  given  value  of  d  there  are  8  values  of  0  in  general  that 
satisfy  the  constraint  that  it  imposes.  Moreover,  observe 
that  if  0  (0  <  0  <  tt/2)  is  a  solution,  then  ir  —  0,  —  tt  4-  0 
&  —0  are  also  solutions.  These  would  correspond  to  the  4 
symmetric  solutions  owing  to  the  inherent  symmetry  in  the 
cone  and  the  ellipse.  Furthermore,  0  and  — ttH-0  correspond 
to  the  same  ellipse  sitting  in  exactly  the  same  position.  In 
many  cases,  some  solutions  for  0  would  be  imaginary  when 
|cos0|  >  1,  meaning  that  there  is  no  feasible  ellipse  for  that 
given  0. 

At  this  stage,  we  have  constrained  the  solution  space  of 
the  eilipse/C-ann  pose,  reducing  the  6-dimensional  pose- 
parameter  space  to  a  function  of  a  single  parameter  0/0* 
With  respect  ro  the  ellipse  frame,  there  are  eight  symmetric 
locations  of  the  X-ray  source,  all  of  them  being  reflections 
along  the  x-y,  y-z  and  z-x  planes.  The  other  eight  solutions 
have  the  same  position  of  the  X-ray  source,  but  the  image 
rotated  by  7r  along  the  center  line  of  each  3D  cone.  Thus  it 
can  be  observed  that,  with  respect  to  the  ellipse,  all  the  fea¬ 
sible  pose  solutions  lie  on  8  non-intersecting  closed  curves 
parametrized  by  (p/0. 

Given  a  (0, 8)  combination,  to  obtain  the  third  rotation 
angle  0  we  apply  the  folio  wing  transformation  to  our  coor¬ 
dinate  frame,  giving  us  a  cone  of  the  form 

XRtReMpRlRjXT  =  XCeXt  =  0  (16) 

where  <7#  is  the  cone  equation  in  a  coordinate  frame  aligned 
with  the  frame  of  the  3D  ellipse,  and  a  rotation  of  0  along  its 
2-axis.  We  next  apply  a  rotation  by  the  angle  0  to  bring  the 
x  and  y  axes  of  our  coordinate  system  parallel  to  the  major 
and  minor  axes  of  the  real  ellipse.  Solving  the  equation,  this 
angle  is  given  by 


0  =  1/2  *  arctan(2  *  BE/(CE  -  AE))  (17) 

The  rotation  transformation  of  0  along  the  z-axis  can  be 
described  as 


Rtjt  — 


cos  0  —  sin  0  0 

sin  0  cos  0  0 

0  0  1 


08) 


The  coordinate  system  created  by  the  combination  of  these 
rotations  now  has  axes  parallel  to  the  axes  of  the  3D  ellipse, 
but  is  centered  at  the  imaging  focal  point.  The  final  step 


therefore,  is  to  compute  the  translation  required  to  bring  the 
center  of  the  frame  to  the  center  of  the  ellipse.  This  transla¬ 
tion  is  computed  by  taking  a  slice  of  the  cone  at  distance  / 
away  from  the  center,  computing  its  major  axis  at  this  point. 
Equating  the  computed  length  of  the  major  axis  as  a  func¬ 
tion  of  /,  to  the  known  value,  provides  /.  The  coordinate 
system  is  then  translated  by  this  vector. 

Combining  these  transformations  allows  us  to  create  an 
algorithm  to  determine  the  set  of  possible  poses  that  could 
have  created  a  given  image.  However,  as  infinitely  many 
possible  (0,0)  combinations  exist,  the  ellipse  image  can¬ 
not  be  used  to  uniquely  determine  the  pose.  It  constrains  5 
of  the  6  degrees  of  freedom  of  pose,  allowing  a  single  de¬ 
gree  of  freedom  that  must  be  removed  through  further  con¬ 
straints. 

2.2.  Combined  Mathematical  Framework 

As  shown  above,  a  single  ellipse  from  each  projection 
image  returns  the  pose  in  (0,  0)  space  with  8  degenerate 
poses,  i.c.,  the  projection  of  the  known  3D  ellipse  in  any  of 
these  poses  would  result  in  exactly  the  same  2D  image.  In 
order  to  eliminate  these  degenerate  cases,  we  propose  to  im¬ 
pose  more  constraints  in  the  form  of  point  correspondences 
across  images.  Even  though  the  projections  of  the  3D  el¬ 
lipse  are  identical,  the  projections  of  the  3D  point  clouds 
differ  and  hence  provides  us  with  extra  constraints  to  break 
the  symmetry  as  shown  below. 

Let  Nm  be  the  number  of  points  identified  in  images 
/m  with  pose  0m  be  the  corresponding  solution 

space  for  C-arm  pose  using  ellipse  from  image  Jm  and  pro¬ 
jection  model  Mm.  Let  be  the  position  of  Ith  segmented 
point  in  mth  image.  When  two  images  are  used,  the  opti¬ 
mization  problem  can  be  formulated  as  follows: 


arg  mm  ^  XT'  r  r 

01,02,/  ljJij 

J£=l  J-l 

where 

E  Af  >lVi;EAi>lVj; fij  e  {0, 1 } 

i= i  1=1 

(19) 

and  C{j  is  the  the  cost  of  matching  point  pn  to  points 
Pj2<  Note  that  it  varies  with  any  variation  in  0\,  $2*  fij 
is  a  discrete  variable  taking  a  value  1/0,  and  deciding  the 
correctness  of  the  match  j),  The  inequalities  force  ev¬ 
ery  segmented  point  to  be  chosen  at  least  once.  Thus,  / 
represents  any  feasible  global  match  (and  vice  versa),  with 
the  cost  of  that  correspondence  given  by  ^  Y^^ijfij*  The 
problem  hence  is  to  compute  ,  #2 ,  /  that  minimize  the  to¬ 
tal  cost  It  should  be  noted  that  since  the  images  represent 
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Figure  2.  Simulated  point  clouds  along  with  ellipse  fiducial  for 
prostate  brachy therapy  and  orthopaedic  procedures 

a  real  situation,  this  optimization  has  a  solution  with  a  near¬ 
zero  cost.  The  procedural  flow  diagram  is  shown  in  Figure 
lb.  For  each  ft*,  there  are  eight  <pi  symmetric  solutions  and 
hence  we  evaluate  the  objective  function  64times  for  two 
images  to  eliminate  the  degenerate  cases.  The  number  of 
evaluations  can  be  decreased  by  tailoring  the  algorithm  to 
be  application  dependent  or  by  constraining  the  motion  of 
the  C-arm  etc.  The  basic  idea  is  that  at  global  optimum,  cor¬ 
rect  pose  would  invoke  correct  correspondences  and  these 
correspondences  would  converge  to  the  optimal  pose.  At 
any  other  point  in  the  solution  space,  incorrect  pose  would 
lead  to  inconsistent  correspondences  and  hence  by  ile ra¬ 
ti  vely  solving  for  both  pose  and  point  correspondences,  a 
unique  global  solution  can  be  achieved. 

In  general,  any  cost  metric  that  globally  measures  the  de¬ 
viation  from  the  observation,  should  perform  well.  We  use 
projection  error  as  our  cost  metric.  This  projection  error 
can  be  computed  as  the  L2  norm  of  the  projection  of  the 
3D  points  (computed  by  intersecting  the  rays  from  the  cor¬ 
responding  2D  points  in  the  X-ray  image  to  its  source)  on 
to  the  2D  images.  Once  the  cost  matrix  is  computed,  we 
use  traditional  Hungarian  assignment  method  to  solve  for 
optimal  point  correspondences  [  ]. 

3,  Experiments  and  Results 

Simulation  Studies:  Two  types  of  synthetic  data,  one  for 
prostate  brachy  therapy  and  the  other  for  orthopaedic  appli¬ 
cations,  were  created  to  evaluate  our  algorithm.  Both  the 
simulation  software  and  the  algorithm  were  implemented  in 
MATLAB,  For  two  images,  the  algorithm  takes  about  20- 
60  seconds  depending  on  the  number  of  points  used,  with 
typical  convergence  in  2-3  iterations.  The  C-arm  imaging 
parameters  for  both  the  simulations  are:  focal  length  =  1000 
mm;  pixel  size  -  0.44  mm/pixel;  image  size  =  (512,  640); 
and  origin  =  (256,  320);  The  diameter  of  the  3D  ellipse  was 
fixed  at  1 8  inches.  For  prostate  brachy therapy  simulations, 
we  created  5  datasets,  each  dataset  has  6  images  taken  al 
regular  intervals  along  a  20°  cone.  Wc  varied  the  imaging 
volume  from  40cc  to  6Qcc  in  increments  of  For  or¬ 
thopaedic  data,  wc  varied  the  imaging  volume  from  lOOcc 
to  lOOOec  in  increments  of  lOOcc.  We  divided  the  experi¬ 
ments  into  three  separate  cloud  sizes:  sparse ,  medium,  and 


Alg.  Performance  lor  small  imaging  volumes  Alg.  Performance  for  large  Imaging  volumes 


Figure  3.  Relative  pose  error  as  a  function  of  the  imaging  volume 
for  small  volumes(left)  and  large  volumes  (right) 

dense  clouds  consisting  of  atleast  10-15,  20-30,  and  70-80 
points  respectively.  These  point  clouds  were  chosen  ran¬ 
domly  for  5  iterations.  Figure  2  shows  an  example  simu¬ 
lated  image  with  ellipse  and  point  clouds,  Wc  believe  that 
this  data  represents  the  spectrum  of  different  types  of  surgi¬ 
cal  scenarios  including  brachy  therapy  and  orthopaedic  pro¬ 
cedures. 

Figure  3  shows  the  simulation  results  for  small  and 
large  imaging  volumes.  Each  datapoint  is  an  average  of 
5  *  *  5  =  375  runs.  Pose  recovery  accuracy  varies  with 

the  size  of  the  point  cloud,  average  being  about  0.34°  for 
small  volumes  arid  0.2°  for  large  volumes.  Although,  the 
variation  is  not  much  significant  with  the  large  volumes  be¬ 
cause  of  the  wide  distribution  of  the  feature  points.  These 
results  show  that  when  there  is  no  error  in  the  data,  pose  can 
be  recovered  accurately. 

We  are  currently  exploring  the  sensitivity  of  the  algo¬ 
rithm  to  the  segmentation  errors  from  both  ellipse  segmen¬ 
tation  and  point  segmentation.  The  overall  failure  rate  for 
large  imaging  volumes  has  decreased  from  10%  to  0.1% 
when  compared  to  the  small  imaging  volumes.  These  sim¬ 
ulation  results  show  that  our  algorithm  successfully  con¬ 
verges  to  the  optimal  solution  in  case  of  large  imaging  vol¬ 
umes  and  small  imaging  volumes  with  sparse  and  medium 
data  distribution  and  is  near  perfect  (success  rate  of  95%) 
for  small  imaging  volumes  with  dense  data  points.  This  can 
be  atiri baled  to  the  fact  that  the  point  mismatches  are  more 
prevalent  in  dense  point  clouds  and  hence  arc  throwing  off 
the  convergence  in  the  two  image  algorithm.  We  show  an 
improvement  in  the  performance  by  using  partial  informa¬ 
tion  from  the  third  image  below. 

Phantom  Experiments:  Experiments  were  performed  on 
a  brachy  therapy  phantom  consisting  oT  both  the  seeds  and 
the  FTRAC  fiducial  rigidly  attached  as  shown  in  Figure  4. 
The  cloud  phantom  comprises  of  multiple  slabs,  thus  ca¬ 
pable  of  multiple  random  point  configurations.  1 00  points 
with  1.56  points  fee  were  used.  X-ray  images  within  a 
20°  cone  around  the  AP-axis  were  randomly  taken  using 
a  Philips  Integris  V3000  11  uo  rose  ope  and  corrected  for  dis¬ 
tortion.  Thus  both  the  seed  locations  and  X-ray  pose  were 
not  biased/optimized  in  any  way,  closely  representing  an 
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Figure  4.  An  image  of  the  point  phantom  attached  to  the  FTRAC 
fiducial  (left),  A  typical  X-ray  image  of  the  combination  (right) 

551 

uncontrolled  surgical  scenario.  Each  image  was  hand  seg¬ 
mented  to  establish  the  true  segmentation  and  correspon¬ 
dence,  thereby  allowing  for  some  segmentation  error.  The 
true  C-arm  pose  and  reconstruction  was  compared  to  that 

556  computed  from  the  algorithm. 

Figure  5.b  shows  the  performance  of  the  algorithm  with 
respect  to  the  number  of  points  used  along  with  the  ellipse. 

559  Each  datapoint  is  an  average  of  5  *  (^)  *  30  =  2250  runs. 
For  phantom  data,  as  few  as  4  —  5  points  along  with  the 
ellipse  are  sufficient  to  accurately  recover  the  pose  (0,74°) 
and  also  are  able  to  attain  a  success  rate  of  99%.  The  sec¬ 
ond  set  of  phantom  experiments  involve  taking  multiple  x- 
ray  images  and  reconstructing  the  3D  seed  locations  from 
its  corresponding  2D  projection  coordinates,  for  which  ac¬ 
curate  C-arm  tracking  is  a  must.  For  each  phantom  dataset, 
we  have  used  sets  of  two  images,  determined  the  relative 
pose  between  these  images  and  then  reconstructed  the  3D 
seed  coordinates.  Although  this  works  perfectly  on  sparse 
data,  our  goal  is  to  make  it  work  on  dense  data  such  as 
brachythcrapy  seed  data.  The  results  arc  shown  in  Table 
1.  Note  that  the  data  in  each  cell  is  an  average  of  15  runs. 
Even  though  the  average  pose  error  is  about  (i°),  only  half 
of  the  seeds  arc  correctly  matched  because  of  the  recon¬ 
struction  singularities  introduced  by  using  only  two  images, 
which  is  fully  expected  for  dense  data. 

To  understand  better,  how  information  from  the  third  im¬ 
age  enhances  the  algorithm  convergence,  we  have  used  a 
third  image  to  identify  the  correct  matches  as  valid  corre¬ 
spondences  between  the  points  identified  by  points  that  cre¬ 
ate  loops  of  size  one.  Given  two  images,  I\ ,  I2 ,  we  select 
a  third  image  /3  and  run  the  algorithm  on  sets  of  two  im¬ 
ages.  Consider  a  point  pa  in  image  / 1  being  matched  to  a 
point  pj2  in  image  I2 ,  and  to  a  point  p^  in  image  I3.  This 
match  between  pn  to  p3 ,2  is  considered  a  valid  match  if  and 
only  if  pj2  matches  p ^3.  The  pose  estimation  algorithm  is 
run  again  using  only  these  valid  matches.  We  refer  to  this 
experiment  as  2.5  images  rather  than  3  images  in  Table  1, 
since  only  partial  information  from  the  third  image  is  used. 

This  added  information  from  the  third  image  has  in¬ 
creased  the  matching  rate  to  90%  with  a  better  pose  and 
reconstruction  accuracy.  The  error  For  mismatched  seeds 
is  still  high.  The  reason  is  that  inspite  of  high  matching 


Aig.  ParformaritG  ,ia  a  function  of  number  of  seeds  used 


Number  of  points  used 

Figure  5.  Performance  as  0  function  of  number  of  points  used 
along  with  the  ellipse.  Note  that  three  to  five  points  achieve  99% 
success  rate. 

rate  and  improved  reconstruction  accuracy,  the  projection 
of  the  solution  using  three  images  is  not  exactly  the  same 
as  that  of  using  two  solutions.  Although  these  accuracies 
are  mostly  acceptable,  prostate  brae hy therapy  needs  99% 
matching  rate  to  determine  the  intra-operative  dosage  ac¬ 
curately,  which  can  be  achieved  by  extending  our  frame¬ 
work  to  use  three  images.  The  basic  idea  here  would  be 
to  use  two  images  to  optimize  the  pose  and  then  use  three 
images  to  solve  for  matching  iteratively.  The  theoretical 
complexity  of  using  three  or  more  images  is  similar  to  that 
of  two  images,  except  that  the  matching  algorithm  would 
be  employing  more  sophisticated  combinatorial  checks  like 
MARSHAL  [  ]. 

4.  Discussion 

A  unified  mathematical  framework  for  C-arm  tracking, 
point  correspondences  and  reconstruction  has  been  pro¬ 
posed,  along  with  experimental  validation  on  simulation 
and  phantom  studies.  Our  experiments  indicate  an  average 
accuracy  of  0,51°  in  orientation,  with  a  STD  of  0.12°,  The 
primary  contribution  of  this  work  is  the  use  of  ellipse  to 
determine  pose  with  a  combinatorial  approach  to  establish 
point  correspondences.  We  show  that  the  projection  of  el¬ 
liptic  curves  constrains  5  out  of  the  6  degrees  of  freedom 
of  the  C-arm  pose.  Furthermore  it  can  also  recover  the  3D 
scale  of  the  reconstruction.  To  completely  recover  the  true 
C-arm  pose,  our  method  uses  constraints  in  the  form  of  ad¬ 
ditional  point  correspondences  from  multiple  images.  We 
show  that  four  or  five  additional  correspondences  can  suc¬ 
cessfully  recover  the  pose  99%  of  the  time  with  an  accuracy 
of  0,74°. 

We  have  conducted  our  validation  on  randomly  selected 
views  and  number/distribution  of  points,  indicating  the  ro¬ 
bustness  of  the  algorithm  to  uniformity  and  reliability  in 
performance.  Though  the  alternatives  of  well  designed  cal¬ 
ibration  fixtures  and  image  based  procedures  arc  also  avail¬ 
able,  they  become  cumbersome  and  impractical,  specifi¬ 
cally  when  employed  in  the  current  clinical  setup.  We  be¬ 
lieve  that  the  presented  framework  represents  a  significant 
step  toward  a  complete  pose  reconstruction  system  employ- 
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Number  of  Seeds 

2  Images  2.5  Images 

40 

55 

70 

85 

100 

40 

55 

70 

85 

100 

Po*se 

Accuracy  (deg) 

0*81 

1,01 

0.95 

1.37 

0,80 

0.49 

0.42 

0.56 

0.59 

0*48 

Matching 

Rate  (%) 

63.38 

62.42 

53.04 

40,94 

45.20 

94.16 

90.25 

90.14 

88.57 

89,37 

Reconstruction 

Error  (matches) (mm) 

3.27 

1.27 

1,72 

3,84 

1.18 

0.19 

0.25 

0,33 

0.22 

0,31 

Reconstruction 

Error  (mismatches)  (nnu) 

26.25 

34.96 

35.4] 

31.04 

33.04 

0.63 

5.04 

5.19 

6.38 

6.06 

Table  1 .  The  algorithm's  performance  on  phantom  data.  2-image  seed  matching  is  completely  erroneous  (left),  with  over  a  quarter  of  the 
seeds  being  mismatched  with  high  reconstruction  error,  yet  low  projection  error  and  reconstruction  accuracy.  Adding  just  little  information 
from  a  third  image  boosts  performance  (right).  Most  seeds  match,  though  the  reconstruction  error  is  high  for  the  few  mismatched  seeds. 


mg  very  little  external  hardware  and  very  minimal  interfer¬ 
ence  with  the  anatomy  being  imaged  (one  could  think  of  a 
small  ellipse  attached  to  a  wire  frame  into  the  field  of  view 
and  imaged  along  with  the  patient).  Moreover,  our  simula¬ 
tion  results  show  that  this  method  can  be  used  for  a  variety 
of  applications  ranging  from  prostate  brachytherapy  (small 
imaging  volume,  dense  point  features)  to  orthopaedic  ap¬ 
plications  (large  imaging  volume,  sparse  to  medium  point 
features). 

Wc  arc  currently  performing  more  simulations  to  vali¬ 
date  the  sensitivity  of  the  algorithm  to  segmentation  errors 
and  arc  also  looking  at  measuring  target  registration  error 
distribution  of  the  point  clouds.  Efforts  to  validate  our  sys¬ 
tem  on  clinical  prostate  brachytherapy  data  are  currently 
underway*  Further  extension  of  this  approach  to  recon¬ 
struct  points  using  three  views  could  significantly  improve 
the  current  clinical  viability  of  intra-operative  quantitative 
fluoroscopy*  especially  in  brachytherapy. 
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ABSTRACT 

We  have  developed  a  tomosynthesis-based  radioactive  seed  localiza¬ 
tion  method  for  prostate  br  achy  therapy.  In  contrast  to  the  projection 
image-based  matching  approach,  our  method  does  not  involve  ex¬ 
plicit  segmentation  of  seeds  and  can  recover  hidden  seeds.  Modified 
distance  map  images  are  computed  from  a  limited  number  of  x-ray 
projection  images,  and  are  backprojected  to  reconstuct  a  3-D  volume 
of  interest.  Candidate  seed  locations  are  extracted  from  the  recon¬ 
structed  volume  and  false  positive  seeds  are  eliminated  by  solving 
an  optimal  geometry  coverage  problem.  The  simulation  results  indi¬ 
cate  that  the  implanted  seed  locations  can  be  estimated  from  three  or 
four  images  depending  on  the  number  of  seeds  if  the  pose  of  a  C-arm 
is  known.  The  algorithm  was  validated  using  phantom  and  clinical 
patient  data. 

Index  Terms —  Tomosynthesis,  prostate  cancer,  brachytherapy, 
modified  distance  map. 

1.  INTRODUCTION 

Prostate  cancer  is  one  of  the  most  leading  cancers  in  men  in  the 
United  States  with  218,890  new  cases  and  27,050  deaths  annually. 
However,  it  can  be  fully  treated  if  detected  early  [1].  Brachytherapy 
is  a  definitive  treatment  for  low  risk  prostate  cancer,  which  involves 
permanent  implantation  of  radioactive  seeds  into  the  prostate.  Its 
success  mainly  depends  on  the  ability  to  implant  a  sufficient  number 
of  seeds  to  ensure  that  the  target  gland  is  treated  with  a  therapeutic 
dose  while  sparing  adjacent  healthy  structures  (e.g.  rectum,  uretha, 
nerve  bundles)  from  excessive  radiation.  Typically,  an  implantation 
plan  is  made  preoperatively  based  on  idealistic  seed  patterns  and  an 
ultrasound  volume  study.  However  the  actual  implant  procedure  in¬ 
troduces  errors  due  to  various  reasons  including  patient  motion,  de¬ 
viation  of  the  needle,  and  prostate  swelling.  In  order  to  overcome 
these  limitations,  intraoperative  planning  under  the  guidance  of  ul¬ 
trasound  and  fluoroscopy  was  proposed. 

There  are  various  computational  tools  available  to  localize  the 
seeds  from  a  limited  number  of  x-ray  images.  Three-dimensional 
coordinates  of  the  implanted  seeds  can  be  calculated  from  multiple 
x-ray  images  upon  resolving  the  correspondence  of  seeds  [2,  3,  4, 
5,  6,  7,  8,  9,  10].  These  methods  require  accurate  segmentation  and 
identification  of  the  seeds  in  all  x-ray  projection  images  followed  by 
solving  matching  problem  between  identified  seeds  in  different  pro¬ 
jection  images.  Incorrect  localization  of  the  seeds  will  cause  unde¬ 
tected  seeds  in  3-D  reconstruction.  Since  typically  up  to  7%  of  the 
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seeds  can  be  hidden  in  the  x-ray  images  [8],  it  is  hard  to  perfectly 
identify  the  seeds  in  all  images  even  if  sophisticated  segmentation 
and  labeling  methods  are  used.  The  unidentified  seeds  are  usually  re¬ 
covered  manually,  which  is  time  consuming  and  sometimes  requires 
extra  x-ray  images.  It  is  sometimes  impossible  to  recover  them  when 
seeds  are  hidden. 

Tutar  et  al.  [11]  proposed  a  modified  tomosynthesis  technique, 
so  called  selective  back  projection  which  requires  large  number  of 
images  (>  7)  and  large  angle  separation  (>  25°).  Also,  it  is  prone 
to  introduce  false  positive  (FP)  seeds  and  to  miss  true  seeds  when 
there  are  C-arm  pose  estimation  errors  (Sec.  2.1  describes  the  reason 
why  pose  estimation  of  the  C-arm  is  necessary).  Their  FP  removal 
process  based  on  the  detected  seed  size  is  also  vulnerable  to  the  pose 
errors  since  these  errors  affect  the  size  of  the  detected  seeds.  A  dis¬ 
crete  tomography  algorithm  based  on  a  distance  map  was  previously 
developed  [12].  However,  some  seeds  (especially  seeds  that  are  in 
a  sparse  seed  region)  were  undetected  when  it  was  applied  to  the 
clinical  patient  data  in  which  seeds  were  smaller  than  those  in  the 
phantom  used  in  [12]  and  pose  errors  were  introduced.  These  algo¬ 
rithms  were  tested  only  on  simulation  and  phantom  data,  but  not  on 
clinical  patient  data. 

In  order  to  make  the  reconstruction  more  robust  to  the  pose  and 
the  calibration  errors,  we  developed  a  tomosynthesis-based  seed  lo¬ 
calization  method  using  shape  control  function-based  modified  dis¬ 
tance  map  images.  We  solve  a  reduced  optimal  coverage  problem  to 
remove  the  FP  seeds.  The  proposed  method  was  tested  on  simulation 
and  phantom  data,  and  was  applied  to  two  clinical  patient  data  sets. 


2.  MATERIALS  AND  METHODS 

Figure  1  shows  an  example  that  can  be  encountered  when  we  re¬ 
construct  the  seeds  using  tomosynthesis.  Three  seeds  (solid  circles) 
are  projected  to  three  images  and  these  three  images  are  backpro¬ 
jected  to  reconstruct  a  3-D  volume.  From  the  reconstructed  volume, 
five  candidate  seeds  are  detected  and  they  appear  to  be  legitimate  in 
every  image.  The  problem  is  to  eliminate  two  FP  seeds  and  detect 
three  true  seeds  correctly.  The  intuition  is  that  each  seed  mark  in 
every  image  must  be  covered  by  at  least  one  of  the  candidate  seeds. 
In  this  simple  example,  there  will  be  seed  marks  in  one  or  more  im¬ 
ages  to  which  no  seed  in  the  volume  is  projected  if  we  eliminate  any 
of  the  true  seeds.  Starting  from  this  intuition,  we  have  developed  a 
theoretical  framework  based  on  optimal  geometry  coverage.  In  order 
to  make  the  reconstruction  more  robust  to  the  pose  and  calibration 
errors,  we  use  a  modified  distance  map  image  instead  of  using  the 
projection  image  itself.  We  now  describe  the  algorithm  in  detail. 


X-ray 

source  'x'o  „  (*  J» 


Fig.  1.  Example  tomosynthesis  geometry. 

2.1.  Image  acquisition,  calibration  and  distortion  correction 

X-ray  projection  images  are  acquired  from  a  C-arm  with  a  limited 
angle  separation,  e.g.  <  20°  due  to  the  limited  space  in  the  op¬ 
erating  room  near  the  patient.  The  acquired  images  are  dewarped 
and  calibrated  (previously)  by  using  a  calibration  phantom.  Since  C- 
arms  in  most  facilities  do  not  have  encoded  rotational  joints  and  they 
may  move  on  wheels  between  shots,  we  need  to  determine  the  pose 
of  the  C-arm  using  tracking  devices.  We  use  a  fluoroscope  tracking 
(FTRAC)  fiducial  which  is  a  radiographic  fiducial  system  creating  a 
unique  projection  image  from  each  direction  [13]. 

2.2.  Seeds  segmentation  and  modified  distance  map  image  com¬ 
putation 

Once  (distortion  corrected)  projection  images  are  acquired,  the  seed 
cloud  region  is  selected  by  user-driven  morphological  operators  and 
then  seed-only  images  are  computed.  Since  the  background  is  highly 
nonuniform  in  actual  patient  images,  the  nonuniform  background  is 
first  extracted  by  dilation  and  then  subtracted  from  the  original  im¬ 
age.  Binary  seed-only  images  are  computed  from  the  background 
subtracted  images  by  adaptive  thresholding.  For  each  seed-only  im¬ 
age,  a  distance  map  is  computed  using  a  distance  transform  which 
computes  the  Euclidean  distance  between  each  pixel  and  its  nearest 
seed  region.  Since  we  do  not  need  to  identify  all  seeds  in  the  2-D  im¬ 
ages,  overlapping  seeds  are  simultaneously  considered  to  be  at  zero 
distance.  Then  the  modified  distance  map  images  are  computed  us¬ 
ing  a  shape  control  function  from  the  distance  maps.  In  this  paper, 
we  use  an  unnormalized  gaussian  as  a  shape  control  function: 

I(x)  =  e-d^)2/2a2  (1) 

where  /(•)  is  the  modified  distance  map  image,  x  G  R2  is  the  pixel 
location  in  the  image,  d  is  the  distance  map  of  the  image,  and  a2 
is  the  variance  which  controls  the  tapering  speed.  The  pixel  value 
inside  the  seed  regions  is  1  and  the  pixel  value  tapers  down  as  the 
distance  between  the  pixel  and  the  nearest  seed  region  increases,  a2 
is  determined  depending  on  the  pose  estimation  errors.  The  intuition 
of  this  approach  is  that  the  seed  regions  in  the  seed-only  image  are 
not  exact  due  to  the  pose  and  calibration  errors  and  the  probability 
that  a  pixel  belongs  to  the  seed  region  is  related  to  the  distance  be¬ 
tween  that  pixel  and  its  nearest  seed  region.  A  pixel  closer  to  the  seed 
region  in  the  seed-only  image  has  higher  probability  that  it  belongs 
to  the  true  seed  region,  but  a  pixel  far  apart  from  the  seed  region  in 
the  seed-only  image  has  small  probability  that  it  belongs  to  the  true 
seed  region. 


2.3.  Volume  reconstruction  and  candidate  seeds  extraction 

A  3-D  volume  is  reconstructed  using  backprojection,  which  is  equiv¬ 
alent  to  a  generalized  form  of  tomosynthesis  for  arbitrary  orienta¬ 
tions.  A  global  reference  coordinate  system  (FTRAC  coordinate 
system)  is  defined  and  a  local  coordinate  system  (C-arm  coordinate 
system)  is  defined  with  respect  to  the  global  reference  coordinates. 
3x4  projection  matrix  which  maps  a  point  in  3-D  space  to  the  cor¬ 
responding  projection  point  in  the  projection  image  at  each  pose  is 
computed  from  the  4  x  4  transformation  between  the  global  refer¬ 
ence  and  the  local  coordinates  and  the  perspective  projection  of  the 
C-arm.  The  projection  matrices  are  used  for  both  reconstruction  and 
cost  computation  for  FP  seeds  removal.  After  the  3-D  reconstruc¬ 
tion,  candidate  3-D  seeds  are  extracted  by  thresholding.  Unlike  a 
reconstruction  from  actual  projection  images,  the  reconstructed  vox¬ 
els  take  values  between  0  and  1  due  to  the  nature  of  the  modified 
distance  map  images,  and  the  threshold  varies  within  a  small  range, 
e.g.  0.9-1. 0,  which  makes  an  automatic  thresholding  based  on  the 
average  size  and  the  number  of  detected  candidate  seeds  possible. 
The  thresholded  3-D  seed  regions  are  labeled  using  connected  com¬ 
ponent  labeling  method  and  the  centroid  of  each  labeled  candidate 
seed  region  is  computed  by  averaging  the  3-D  coordinates  of  the 
voxels  in  each  seed  region. 

2.4.  False  positive  seeds  removal 

A  tomosynthesis-based  approach  is  prone  to  introduce  FP  seeds. 
Therefore  an  FP  seed  elimination  process  from  the  candidate  seeds 
is  necessary.  This  problem  is  solved  as  a  reduced  optimal  geometric 
coverage  problem  as  was  done  in  [12],  and  a  detailed  description  is 
followed  for  the  completeness  of  the  paper. 

The  goal  in  this  process  is  to  find  the  Nt  true  seeds  from  Nc 
candidate  seeds  such  that  all  the  2-D  seed  regions  are  covered  in  all 
projection  images.  Since  an  FP  seed  is  projected  close  to  some  true 
seeds  in  every  image  but  true  seeds  are  not  projected  close  to  the 
other  true  seeds  in  all  the  images,  a  cost  function  of  a  given  seed 
can  be  defined  as  the  sum  of  the  closest  distances  between  the  pro¬ 
jections  of  this  seed  and  the  projections  of  all  the  other  true  seeds 
in  all  images.  However,  since  this  optimal  coverage  problem  is  NP- 
hard  [14]  and  it  requires  impractical  amount  of  time  for  a  large  num¬ 
ber  of  seeds,  a  greedy  search  to  minimize  local  costs  rather  than  the 
global  cost  is  adopted  to  reduce  the  computational  burden. 

To  ensure  that  the  projections  of  selected  true  seeds  cover  all  the 
2-D  seed  regions  in  all  seed-only  images,  all  2-D  seed  regions  in  all 
seed-only  images  are  labeled  using  connected  component  labeling 
method  and  the  candidate  seeds  are  clustered  based  on  their  pro¬ 
jections  in  each  labeled  seed-only  image.  The  purpose  of  the  seed 
labeling  is  not  to  segment  the  2-D  seed  regions  or  to  identify  the 
number  of  true  seeds  in  each  image,  so  the  overlapped  seed  regions 
will  be  labeled  as  one.  This  clustered  seed  labels  are  used  for  the 
coverage  function  minimization. 

All  candidate  seeds  are  first  projected  onto  each  image.  Let  xn 
be  a  3-D  candidate  seed  position  and  Pl  be  a  projection  matrix  for 
the  ith  projection  image.  Then  the  projection  of  xn  to  the  ith  image 
becomes  Plxn.  Let  Lz(x)  be  a  label  corresponding  to  a  point  x, 
then  the  label  of  a  projection  of  a  3-D  seed  position  xn  in  the  ith 
image  becomes  Ll(Plxn).  The  projections  of  all  candidate  seeds 
in  the  ith  image  are  clustered  as  sets  —  {xn\ Lz(Pzxn)  —  k}, 
k  —  1,  2, 3 ,  ...Kl  where  K%  is  the  number  of  labels  in  the  ith  la¬ 
beled  seed-only  image.  The  seed  region  with  label  k  in  ith  image 
is  covered  by  1 1 Qlk \ |  seeds  where  \\Ct%\\  is  the  cardinal  of  set 
and  | \n\ 1 1  >  1.  If  a  seed  region  is  covered  by  only  one  seed,  i.e. 
||f4||  =  1,  this  seed  must  be  a  true  seed  because  otherwise  this 


region  cannot  be  covered.  The  set  of  such  seeds  can  be  described  as: 
G  =  Uiix^ViP^n)  =  k  and  ||fi*fc||  =  l,k  =  1,2, 

(2) 

for  i  =  1,  2, Np  where  Np  is  the  number  of  projection  images. 
The  optimization  problem  can  be  reduced  to  choose  (Nt  —  ||G||) 
seeds  from  (Nc  —  |  |G|  |)  candidate  seeds.  And  a  local  cost  function 
is  defined  as: 

Np 

C(xn)  =  -  ^  1  +  f  3fK) ,  for  xn  €  (S  -  G)  (3) 

i=l  l  +  dn 

D\xn)  =  min \\P'lxrn  —  Plxn\\,  for Xm  £  S 

m^n 

where  S  is  a  set  of  candidate  seeds,  and  d\  is  the  distance  from  the 
projection  of  xn  to  the  nearest  seed  region  in  the  ith  image,  d ln  is 
included  in  the  cost  function  to  reflect  the  effect  of  imperfect  pose 
estimation.  This  problem  is  solved  using  greedy  search  iteratively. 
During  each  iteration,  a  seed  that  has  the  largest  cost  value  computed 
by  (3)  is  considered  as  an  FP  seed  and  is  removed  from  S  and  G  is 
updated  at  each  iteration  if  there  are  additional  seeds  which  cover 
some  seed  regions  alone  after  removing  one  FP  seed.  Iteration  con¬ 
tinues  until  Nt  seeds  are  left. 

3.  NUMERICAL  RESULTS 


3.1.  Simulations 

Simulation  studies  were  conducted  on  synthetic  projection  images 
to  evaluate  the  performance  of  our  method.  Three  cases  were  con¬ 
sidered  for  a  50  cc  prostate:  (1)  seed  density  of  1.5  seeds/cc  with 
72  implanted  seeds,  (2)  seed  density  of  2.0  seeds/cc  with  96  im¬ 
planted  seeds,  and  (3)  seed  density  of  2.25  seeds/cc  with  112  im¬ 
planted  seeds.  Three  data  sets  were  generated  for  each  case.  The  fo¬ 
cal  length  of  the  C-arm  and  the  pixel  size  of  the  image  were  chosen 
to  be  1000  mm  and  0.442  mm2,  respectively  based  on  our  experi¬ 
ence.  Each  seed  is  represented  by  a  cylinder  with  radius  of  0.4  mm 
and  length  of  1 .45  mm.  In  each  data  set,  there  are  six  images  gen¬ 
erated  on  a  20°  cone  around  the  AP  axis.  No  C-arm  pose  error  was 
introduced.  3-D  reconstructions  of  the  seeds  are  computed  based  on 
three  and  four  images  with  a  voxel  size  of  0.53  mm3.  We  have  cho¬ 
sen  three  combinations  of  selecting  three  (or  four)  images  out  of  six 
avaliable  images  in  each  data  set  and  computed  nine  reconstructions 
for  each  case  (3  combinations  x  3  data  sets).  Since  there  was  no  pose 
error,  a  was  chosen  to  be  small  (e.g.  1  pixel  =  0.5  mm)  and  threshold 
for  localizing  candidate  seeds  in  3-D  reconstruction  was  chosen  to  be 
high  (e.g.  1.0).  The  estimated  seed  positions  were  compared  to  the 
ground  truth.  The  results  are  shown  in  Table  1  and  imply  that  the  im¬ 
planted  seeds  can  be  localized  with  a  detection  accuracy  of  >  98% 
or  >  99%  from  three  or  four  images,  respectively,  depending  on  the 
number  of  seeds  when  there  is  no  C-arm  pose  error. 

3.2.  Phantom  experiments 

Phantom  experiments  were  performed  on  a  seed  phantom  which  was 
fabricated  using  acetol.  The  phantom  consists  of  twelve  slabs  with 
thickness  of  5  mm  and  each  slab  has  at  least  a  hundred  holes  with  5 
mm  spacing  where  seeds  can  be  positioned.  The  FTRAC  was  pre¬ 
cisely  attached  to  the  seed  phantom  in  a  known  position.  There  was 
about  0.5°-  1°  rotational  error  introduced  in  the  attachment  process 
which  results  in  about  0.5  mm  error  in  the  ground  truth  seed  loca¬ 
tions.  Three  data  sets  were  generated  with  the  numbers  of  seeds  of 
42,  72,  and  102  keeping  seed  density  constant  at  about  1.56  seeds/cc 


Fig.  2.  (a)  Example  clinical  patient  image  showing  66  seeds  and 
FTRAC  images,  (b)  The  estimated  seed  centroids  projected  onto 
one  of  three  projection  images  used  for  reconstruction. 


for  all  three  data  sets.  For  each  data  set,  six  images  were  acquired 
within  a  20°  cone  around  the  AP  axis  using  a  Philips  Integris  V3000 
fluoroscope  and  were  dewarped  using  previously  acquired  calibra¬ 
tion  phantom  data.  Four,  five,  and  six  images  were  selected  from 
the  six  available  images  in  each  data  set  based  on  the  residual  er¬ 
rors  provided  by  the  FTRAC  software,  and  were  used  for  3-D  re¬ 
constructions.  The  voxel  size  of  the  3-D  reconstructions  was  0.53 
mm3.  Considering  the  pose  errors,  a  was  chosen  to  be  larger  (e.g. 
3  pixels  =  1.5  mm)  and  the  threshold  for  localizing  candidate  seeds 
in  3-D  reconstruction  was  chosen  to  be  smaller  (e.g.  0.95)  than  the 
values  chosen  for  simulations.  The  estimated  seed  positions  were 
compared  to  the  ground  truth,  and  the  results  are  shown  in  Table  2. 
The  results  imply  that  the  implanted  seed  locations  can  be  estimated 
with  a  detection  accuracy  of  >  97%  from  five  projection  images. 

3.3.  Clinical  experiments 

The  proposed  method  was  applied  to  two  clinical  patient  data  sets. 
X-ray  images  were  taken  within  a  10°  cone  around  the  AP  axis  us¬ 
ing  OEC  9800  fluoroscope,  and  were  dewarped  and  calibrated  using 
calibration  phantom  data  acquired  prior  to  the  surgery.  The  FTRAC 
from  which  the  pose  of  the  C-arm  at  each  image  was  estimated  was 
precisely  attached  to  the  needle  insertion  template  in  a  known  posi¬ 
tion.  61  and  66  103Pd  seeds  were  implanted  into  30  cc  prostate  and 
the  size  of  the  seed  was  4.92  mm  (length)  and  1.0  mm  (radius)  with 
radio-opaque  size  of  about  1.45  mm  (length)  and  0.8  mm  (radius). 
Three  images  were  selected  from  the  acquired  images  (10  images  for 
61  seeds,  9  images  for  66  seeds)  based  on  the  residual  errors  com¬ 
puted  by  the  FTRAC  software.  The  a  =  5  pixels  =  2.5  mm  and  the 
threshold  =  0.98  for  localizing  the  candidate  seeds  were  used.  Since 
the  exact  locations  of  the  seeds  were  unknown,  the  correspondence 
between  the  projection  of  the  estimated  seeds  and  the  actual  seeds 
in  the  images  was  visually  assessed.  The  estimated  seed  locations 
were  also  compared  with  those  computed  by  an  existing  software, 
MARSHAL  [7]  and  the  differences  were  computed.  In  Figure  2  are 
shown  an  example  patient  image  and  a  magnified  image  onto  which 
the  centroids  of  the  estimated  seeds  are  projected.  All  seeds  were 
well  matched  for  both  cases  and  the  results  are  shown  in  Table  3. 

4.  CONCLUSIONS 

A  novel  method  for  prostate  brachytherapy  seed  reconstruction 
using  C-arm  images  was  described.  Modified  distance  map  im¬ 
ages  are  generated  from  2-D  projection  images  and  are  used  for 
tomosynthesis-based  3-D  reconstruction.  True  seed  locations  are 


Table  1.  Simulation  results  using  different  number  of  images  for  different  data  sets. 


Number  of  true  seeds 

72 

96 

112 

Number  of  images  used 

3 

4 

3 

4 

3 

4 

Number  of  candidate  seeds 

73.6 

72.0 

104.9 

97.1 

121.7 

113.9 

Correctly  estimated  seeds  (%) 

100 

100 

98.3 

100 

98.9 

99.6 

Mean  reconstruction  error  (mm) 

0.61 

0.57 

0.64 

0.58 

0.65 

0.57 

STD  of  reconstruction  errors  (mm) 

0.14 

0.09 

0.31 

0.11 

0.24 

0.11 

Table  2.  Phantom  experiment  results  using  different  number  of  images  for  different  data  sets. 


Number  of  true  seeds 

42 

72 

102 

Number  of  images  used 

4 

5 

6 

4 

5 

6 

4 

5 

6 

Number  of  candidate  seeds 

43 

42 

42 

75 

72 

72 

117 

112 

107 

Correctly  estimated  seeds  (%) 

100 

100 

100 

98.6 

100 

100 

93.1 

97.1 

97.1 

Mean  reconstruction  error  (mm) 

1.75 

1.26 

1.05 

1.04 

0.67 

0.65 

1.27 

1.50 

0.98 

STD  of  reconstruction  errors  (mm) 

0.64 

0.52 

0.42 

0.44 

0.41 

0.37 

0.55 

0.46 

0.50 

Table  3.  Clinical  experiment  results. 


Number  of  true  seeds 

61 

66 

Number  of  images  used 

3 

3 

Number  of  candidate  seeds 

68 

80 

Correctly  matched  seeds  (%) 

100 

100 

Mean  reconstruction  difference  (mm) 

0.96 

1.04 

STD  of  reconstruction  difference  (mm) 

0.61 

0.72 

separated  from  a  set  of  candidate  seeds  detected  from  the  reconstruc¬ 
tion  by  solving  optimal  coverage  problem.  This  method  requires 
slightly  larger  number  of  images  compared  to  the  2-D  image-based 
approach,  but  the  attractive  feature  of  the  proposed  method  is  that  it 
can  recover  all  the  seeds  automatically,  including  the  hidden  seeds, 
without  an  exact  seed  identification.  Through  simulations,  phantom, 
and  clinical  studies,  our  method  was  evaluated,  successfully  local¬ 
izing  the  implanted  seeds.  Further  validation  on  more  clinical  data 
will  be  shown  in  a  future  publication. 
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ABSTRACT 

Purpose:  Brachytherapy  (radioactive  seed  insertion)  has  emerged  as  one  of  the  most  effective  treatment  options 
for  patients  with  prostate  cancer,  with  the  added  benefit  of  a  convenient  outpatient  procedure.  The  main 
limitation  in  contemporary  brachytherapy  is  faulty  seed  placement,  predominantly  due  to  the  presence  of  intra¬ 
operative  edema  (tissue  expansion).  Though  currently  not  available,  the  capability  to  intra-operatively  monitor 
the  seed  distribution,  can  make  a  significant  improvement  in  cancer  control.  We  present  such  a  system  here. 

Methods:  Intra-operative  measurement  of  edema  in  prostate  brachytherapy  requires  localization  of  inserted 
radioactive  seeds  relative  to  the  prostate.  Seeds  were  reconstructed  using  a  typical  non-isocentric  C-arm,  and 
exported  to  a  commercial  brachytherapy  delivery  system.  Technical  obstacles  for  3D  reconstruction  on  a  non- 
isocentric  C-arm  include  pose-dependent  C-arm  calibration;  distortion  correction;  pose  estimation  of  C-arm 
images;  seed  reconstruction;  and  C-arm  to  TRUS  registration. 

Results:  In  precision-machined  hard  phantoms  with  40-100  seeds  and  soft  tissue  phantoms  with  45-87  seeds, 
we  correctly  reconstructed  the  seed  implant  shape  with  an  average  3D  precision  of  0.35  mm  and  0.24  mm, 
respectively.  In  a  DoD  Phase- 1  clinical  trial  on  6  patients  with  48-82  planned  seeds,  we  achieved  intra-operative 
monitoring  of  seed  distribution  and  dosimetry,  correcting  for  dose  inhomogeneities  by  inserting  an  average  of 
4.17  (1-9)  additional  seeds.  Additionally,  in  each  patient,  the  system  automatically  detected  intra-operative  seed 
migration  induced  due  to  edema  (mean  3.84  mm,  STD  2.13  mm,  Max  16.19  mm). 

Conclusions:  The  proposed  system  is  the  first  of  a  kind  that  makes  intra-operative  detection  of  edema  (and 
subsequent  re-optimization)  possible  on  any  typical  non-isocentric  C-arm,  at  negligible  additional  cost  to  the 
existing  clinical  installation.  It  achieves  a  significantly  more  homogeneous  seed  distribution,  and  has  the  potential 
to  affect  a  paradigm  shift  in  clinical  practice.  Large  scale  studies  and  commercialization  are  currently  underway. 

Keywords:  Fluoroscopic  guidance,  Prostate  brachytherapy,  Low  cost,  Edema. 

1.  INTRODUCTION 

With  an  annual  incidence  of  over  200,000  new  cases  &  33,000  deaths  in  the  US,  prostate  cancer  continues  to  be 
the  most  common  cancer  in  men,1  and  is  expected  to  double  its  incidence  rates  by  2015.  Currently  one  in  every 
six  men  get  diagnosed  with  it  at  some  point  in  their  life.  For  several  decades,  the  definitive  treatment  for  low  risk 
prostate  cancer  was  radical  prostatectomy  or  external  beam  radiation  therapy  (EBRT).  However,  developments 
in  prostate  brachytherapy  have  seen  it  emerge  as  a  primary  treatment  option,  with  recent  studies  showing  an 
overall  15-year  survival  rate  of  74%  (88%,  80%  &  53%  in  low,  intermediate  &  high-risk  patients).  Improved  pa¬ 
tient  selection  and  implant  quality  has  resulted  in  multiple  brachytherapy  reports  demonstrating  survival  rates 
equivalent  to  the  best  published  radical  prostatectomy  and  EBRT  outcomes.2  This  has  lead  to  a  large  increase 
in  the  number  of  brachytherapy  procedures,  with  30%-40%  of  all  prostate  cancer  patients  receiving  brachyther¬ 
apy  as  part  of  their  treatment,3  while  utilization  rates  for  prostatectomy  and  external-beam  radiotherapy  have 
fallen  by  over  10%.  It  is  currently  estimated  that  at  over  50,000  brachytherapy  procedures  are  performed  annually. 

Brachytherapy  is  a  convenient  one-time  outpatient  procedure,  without  the  surgical  trauma  of  prostatectomy 
or  the  grinding  routine  of  EBRT.  In  this  treatment  a  number  of  small  (~  1x5  mm)  radioactive  capsules  are 
implanted  into  the  prostate  to  kill  the  cancer  by  emitting  radiation.  It  was  first  performed  in  1911  with  limited 


success.  In  the  1960’s,  the  use  of  Au-198  &  1-125  isotopes  triggered  modern  day  permanent  seed  implantation. 
Many  new  techniques  were  introduced,  but  their  limitations  prevented  widespread  adoption.  Nevertheless,  it 
was  found  that  disease  control  was  better  in  early-stage  patients  who  received  high-quality  implants  (60%  versus 
20%  local  control).3  The  advent  of  transrectal  ultrasound  (TRUS)-guided  procedures  in  the  1980’s,  along  with 
computerized  planning,  lead  to  an  increase  in  seed  placement  accuracy  and  a  superior  distribution  of  the  seeds 
throughout  the  prostate.  Early  results  showed  that  patients  treated  between  1988-90,  performed  better  than 
those  treated  by  the  same  physicians  between  1986-87,  the  only  difference  being  the  quality  of  the  implant.  This 
further  supported  the  hypothesis  that  higher- quality  implants  resulted  in  better  outcomes.  As  the  procedure 
has  become  more  and  more  popular,  many  technical  improvements  have  been  added  (eg.  inverse  planning  and 
post-op  CT)  to  improve  the  implant  quality. 


With  virtually  all  studies  agreeing  that  a  high  quality  implant  is  the  key  to  successful  outcomes,  the  next 
paradigm  shift  is  expected  to  be  the  capability  to  intra-operatively  monitor  the  inserted  seeds.  The  biggest 
bottleneck  toward  this,  is  seed  migration  induced  by  edema.  This  causes  insufficient  dose  to  the  cancer  and/or 
excess  radiation  to  the  rectum,  urethra,  and  bladder.  Edema  is  a  natural  response  of  the  body,  wherein  the 
tissue  expands  due  to  the  needle-insertions  and  radiation.  Studies  have  shown  that  edema  could  increase  the 
prostate  volume  by  up  to  100%,  decreasing  exponentially  with  time  (mean  half-life  of  9.3  days),  and  potentially 
underdosing  some  patients  by  as  much  as  32%. 4  A  small  fraction  of  brachytherapy  patients  suffer  from  adverse 
side-effects  like  secondary  malignancies  in  the  rectum  &  bladder(7.6%5).  Minor  side-effects  include  temporary 
urinary  symptoms  (50-80%6),  rectal  bleeding  (2-5%3)  and  sexual  impotency  (20%3).  All  these  side-effects  stem 
from  imperfect  execution  of  the  intended  plan,  mostly  due  to  edema.  Thus,  with  the  emphasis  shifting  to  quality 
of  life  after  treatment,  intra-operative  monitoring  of  seed  distibution  has  become  the  holy  grail. 


In  order  to  avoid  over /under- dosing,  the  seed  locations  need  to  be  determined  intra-operatively,  which  is 
difficult,  owing  to  significant  seed  migration  due  to  edema.  Contemporary  research  has  tried  to  monitor  the 
seeds  from  TRUS  images  by  linking  seeds  with  spacers,7  using  X-rays  to  initialize  segmentation,8  using  vibro- 
acoustography,9  transurethral  ultrasound10  as  a  new  imaging  modality,  or  segmenting  them  directly  in  TRUS 
images,11  sometimes  by  using  specially  manufactured  corrugated  seeds  that  are  better  visible  than  conventional 
ones.12  But  even  when  meticulously  hand-segmented,  up  to  25%  of  the  seeds  remain  hidden  in  TRUS.13  The 
seeds  have  excellent  visibility  under  X-rays,  with  C-arms  being  ubiquitous  (60%  availability)  in  brachytherapy  for 
gross  visual  assessment.14  In  spite  of  significant  efforts  that  have  been  made  towards  computational  fluoroscopic 
guidance  in  general  surgery,15  C-arms  cannot  yet  be  used  for  intra-operative  brachytherapy  guidance  due  to  a 
plethora  of  technical  limitations.  While  several  groups  have  published  protocols  and  clinical  outcomes  favorably 
supporting  C-arm  fluoroscopy  for  intra-operative  dosimetric  analysis,16-18  this  technique  is  yet  to  become  a 
standard  of  care  across  hospitals.  The  most  important  current  limitation  towards  a  standard  of  care  system,  is 
the  inability  to  use  a  non-isocentric  C-arm,  the  type  most  commonly  found  in  hospitals. 


To  achieve  intra-operative  monitoring  in  prostate  brachytherapy,  we  report  a  new  system  that  reconstructs 
3D  seed  locations  (visible  in  X-ray)  and  spatially  registers  them  to  the  prostate  (visible  in  TRUS).  We  discuss 
briefly  the  variety  of  technical  issues  in  Section  2,  following  up  with  results  from  phantom  experiments  &  a 
Phase-I  clinical  trials  in  Section  3.  Our  primary  contribution  to  the  state  of  the  art  is  our  ability  to  use  any 
typical  non-isocentric  uncalibrated  C-arm  present  in  most  hospitals.  We  believe  that  the  availability  of  this 
technology,  followed  up  with  large  scale  clinical  studies  and  commercialization  (ongoing),  promises  to  lead  to  a 
paradigm  shift  in  the  standard  of  care  for  image-guided  prostate  brachytherapy. 

2.  METHODS  AND  MATERIALS 

The  system  is  designed  to  easily  integrate  easily  with  commercial  brachytherapy  installations.  We  employ  a 
regular  clinical  brachytherapy  setup,  without  alteration,  including  a  treatment  planning  workstation  &  stabi- 
lizer/stepper  (Interplant®,  CMS,  St  Louis),  TRUS  (B&K  Medical  Pro  Focus)  and  a  C-arm  (GE  OEC  9600/9800). 
The  C-arm  is  interfaced  with  a  laptop  through  an  NTSC  video  line  and  frame  grabber,  making  the  image  capture 
independent  of  the  C-arm  model. 
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Figure  1.  Overview  of  the  proposed  solution.  The  FTRAC  fiducial  tracks  C-arms,  and  also  registers  TRUS  to  C-arm 
images,  making  quantitative  brachytherapy  possible. 


Workflow:  The  clinical  workflow  (Fig.  1)  is  identical  to  the  standard  procedure  until  the  clinician  decides  to 
run  a  reconstruction  and  optimization.  A  set  of  C-arm  images  are  collected  with  a  separation  as  wide  as  clinically 
possible  (usually  10°  around  AP-axis)  and  synchronously  transferred  to  the  laptop.  After  processing  the  images, 
the  seeds  are  reconstructed  and  their  3D  locations  exported  to  the  Interplant®  system.  The  physician  uses 
standard  Interplant®  tools  to  analyze,  optimize  and  modify  the  remainder  of  the  plan.  The  procedure  concludes 
when  the  exit  dosimetry  shows  no  cold  spots  (under-radiated  locations). 

Numerous  technical  obstacles  have  to  be  overcome  to  realize  C-arm  based  intra-operative  dosimetry:  (a) 
pose  estimation  of  C-arm  images;  (b)  C-arm  calibration;  (c)  image  distortion  correction;  (d)  seed  segmentation; 
(e)  seed  matching  &  reconstruction;  (f)  registration  of  C-arm  to  TRUS;  (g)  dosimetry  analysis;  and  finally  (h) 
implant  optimization.  We  have  developed  a  system  that  overcomes  these  limitations  in  providing  quantitative 
intra-operative  dosimetry.  In  what  follows,  we  will  describe  briefly  each  component  of  the  system,  skipping  the 
detailed  mathematical  framework. 


Pose  Estimation:  The  most  critical  component  of  a  clinically  usable  solution  for  3D  fluoroscopic  guidance  is 
C-arm  pose  estimation.  C-arms  available  in  most  hospitals  do  not  have  encoded  rotational  joints,  making  the 
amount  of  C-arm  motion  unavailable.  C-arm  tracking  using  auxiliary  trackers  is  expensive,  inaccurate  in  the 
presence  of  metal  (EM  tracking)  or  intrudes  in  the  operating  room  (optical  tracking).  There  has  been  some  prior 
work  on  fiducial  based  tracking,  wherein  a  fiducial  is  introduced  in  the  X-ray  FOV  and  its  projection  in  the  image 
encodes  the  6  DOF  pose  of  the  C-arm.  The  most  significant  problem  in  contemporary  designs  is  that  though 
sufficiently  accurate,  they  are  too  bulky  to  be  easily  used  in  a  clinical  setting.  To  improve  clinical  usability,  the 
size  of  the  fiducial  can  be  decreased,  usually  at  a  significant  expense  of  the  accuracy. 

We  proposed  a  new  fluoroscope  tracking  fiducial  design,  FTRAC,  that  uses  an  ellipse.19  The  fiducial  is 
illustrated  in  Figure  2,  5.  The  ellipse  makes  pose  recovery  accurate  by  (a)  always  projecting  as  an  ellipse;  (b) 
allowing  an  accurate  segmentation;  and  (c)  providing  closed  form  Jacobian  formulations  for  fast  optimization. 
The  FTRAC  design  has  salient  features,  including  small  dimensions  (3x3x5cm),  no  special  proximity  requirements 
to  the  anatomy,  and  is  relatively  inexpensive.  In  particular,  the  small  size  makes  it  easier  to  be  always  in  the 
FOV  &  to  be  robust  to  image  distortion.  Extensive  phantom  experiments  indicated  a  mean  tracking  accuracy  on 
distorted  C-arms  of  0.56  mm  in  translation  and  0.19°  in  rotation,  an  accuracy  comparable  to  expensive  external 
trackers. 


Figure  2.  Images  of  the  FTRAC  fiducial  (a)  wire  model;  (b)  photograph;  (c)  X-ray  image 


C-arm  Source  Calibration  and  Image  Distortion:  Since  both  C-arm  calibration  and  distortion  are  pose- 
dependent,  contemporary  fluoroscopy  guidance  systems  calibrate/  distortion-correct  at  each  imaging  pose.  This 
is  done  using  a  cumbersome  calibration- fixture,  which  is  a  significant  liability.  Our  approach  is  a  complete 
departure.  Using  a  mathematical  &  experimental  framework,  we  demonstrated  that  calibration  is  not  critical  for 
prostate  seed  reconstruction;  i.e.  just  an  approximate  pre-operative  calibration  suffices.  The  central  intuition  is 
that  object  reconstruction  using  a  mis-calibrated  C-arm  changes  only  the  absolute  positions  of  the  objects,  but 
not  their  relative  ones  (Fig.  3).  Additionally,  statistical  analysis  of  the  distortion  on  a  GE  OEC  9600,  inside  a 
15°  limited  workspace  revealed  that  just  a  single  pre-operative  correction  at  the  AP-axis  can  reduce  the  average 
distortion  in  the  image  from  3.31  mm  to  0.51  mm.  These  errors  are  expected  to  be  similar  for  most  modern 
C-arms,  with  a  more  involved  discussion  available  in.20  This  residual  amount  of  distortion  in  the  X-ray  image 
is  acceptable  for  accurate  3D  reconstruction,  especially  when  used  with  image  based  C-arm  tracking. 


Figure  3.  (a)  Mis-calibration  conserves  relative  reconstruction  between  objects  A  and  B  (eg.  seeds),  (b)  X-ray  image 
before  segmentation  (left).  There  is  also  the  FTRAC  in  the  image.  The  image  after  segmentation  (right);  the  blue  ’+’ 
symbol  represent  individual  seeds  and  the  blue  indicates  a  seed  that  is  a  part  of  a  multiple  seed  cluster. 


Seed  Segmentation:  Seed  segmentation  in  C-arm  video  images  is  usually  hindered  by  poor  image  resolution. 
We  developed  an  automated  seed  segmentation  algorithm  that  employs  the  morphological  top-hat  transform 
to  perform  the  basic  seed  segmentation,  followed  by  thresholding,  region  labeling,  and  finally  a  statistical  clas¬ 
sification  into  two  classes  -  single  seeds  and  clusters  containing  overlapping  seeds.  Using  some  simple  ellipse 
partitioning  techniques,  these  clustered  seeds  are  finally  broken  down  into  single  seeds.  The  algorithm  was  tested 
on  clinical  images  and  found  to  be  sufficiently  accurate.  Out  of  total  763  seeds  from  10  patient  images  of  Palla¬ 
dium  implants,  758  seeds  were  correctly  identified;  only  2  seeds  clusters  were  not  correctly  resolved  and  2  false 
positives  were  added.  The  segmentation  is  verified  on  the  screen  to  allow  for  a  manual  bypass  by  the  surgeon. 


Seed  Correspondence  &  Reconstruction:  The  3D  coordinates  of  the  implanted  seeds  can  now  be  triangu¬ 
lated  by  resolving  the  correspondence  of  seeds  in  the  multiple  X-ray  images.  As  explained  in21  we  formalized  the 
seed  correspondence  as  a  combinatorial  optimization  problem.  As  seen  in  Figure  4,  a  network  is  created  where 
any  flow  represents  a  matching,  the  desired  solution  being  the  flow  with  minimum  cost,  computed  using  the 
cycle-canceling  algorithm.  Our  formulation  has  many  preferred  features:  (a)  exact  solutions;  (b)  claims  on  the 
problem  complexity;  and  (c)  optimality  considerations.  We  showed  that  a  polynomial-time  perfect  solution  is 
not  achievable  and  proposed  a  practical  solution  (MARSHAL)  that  runs  practically  in  0(N 3)  using  any  number 
of  images,  where  N  is  the  number  of  inserted  seeds.  In  comparison,  previous  solutions  have  predominantly  been 
heuristic  explorations  of  the  large  search  space.  In  addition,  the  framework  robustly  resolves  all  the  seeds  that 
are  hidden  in  the  images  (as  many  as  7%  can  be  hidden  due  to  the  high  seed  density).  A  simple  extension  to 
the  network  at  strategic  nodes,  allowing  a  flow  of  2  units  instead  of  1,  enables  MARSHAL  to  also  automatically 
recover  hidden  seeds.  MARSHAL  typically  reconstructs  99.8%  of  the  seeds  and  runs  in  under  5s  in  MATLAB, 
exhibiting  a  significantly  higher  than  the  required  minimum-detection-rate  of  95%  that  has  been  suggested.22 
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Figure  4.  The  network  flow  formulation  used  to  solve  the  hidden-seed  correspondence  problem. 


Registration  of  C-arm  to  TRUS:  The  3D  seed  coordinates  are  reconstructed  with  respect  to  the  FTRAC 
and  need  to  be  registered  to  the  prostate  boundaries  visible  in  the  TRUS  images.  This  is  a  difficult  problem, 
owing  to  the  lack  of  physical  access  to  the  patient-anatomy  and  the  inherently  complementary  natures  of  the 
two  modalities.  X-rays  predominantly  images  hard  tissue,  while  Ultrasound  looks  at  soft  tissue  boundaries.  In 
a  quick  survey  of  X-ray  to  TRUS  registration,  various  groups  have  attempted  to  use  catheters,  gold  marker 
seeds,  needles  or  the  inserted  radioactive  seeds  as  markers.  Unfortunately,  implanted  markers  in  Ultrasound  are 
very  hard  to  differentiate  from  various  other  objects  like  inserted  seeds  &  needle  tracts.  Moreover,  they  are  also 
susceptible  to  intra-operative  tissue  expansion,  making  the  registration  unreliable.  Alternately,  radio-opaque 
beads  were  attached  to  the  probe  and  have  been  used  for  registration,  requiring  a  permanent  alteration  to  the 
probe,  which  for  many  practitioners  is  not  desirable.  Altogether,  a  comprehensive  and  clinically  reliable  solution 
for  spatial  registration  of  fluoroscopy  and  TRUS  images  is  currently  not  known. 


Figure  5.  The  FTRAC  fiducial  &;  the  needle  insertion  template  can  be  pre-calibrated  using  a  rigid  mount. (a)  A  CAD 
model  of  the  FTRAC  fiducial  mounted  on  the  seed-insertion  needle  template  using  a  rigid  connector,  (b)  An  actual 
photograph  of  the  FTRAC  mounted  on  the  template. (c)  A  zoomed  annotated  photograph  of  the  clinical  setup. 


In  commercial  brachytherapy  systems  the  needle  insertion  template  is  already  pre-registered  to  TRUS  probe 
as  per  the  FDA  approved  clinical  protocol.  Hence,  we  propose  to  mount  the  FTRAC  fiducial  on  top  of  this 
template  in  a  known  rigid  position,  using  a  precision-machined  mechanical  connector.  The  arrangement  is 
illustrated  in  Figure  5.  Thus  a  simple  application  of  the  various  known  frame  transformations,  registers  the  3D 
seeds  (FTRAC)  to  the  prostate  (TRUS). 


System  Implementation:  We  have  integrated  the  previously  discussed  functions  into  a  complete  MATLAB 
program  with  a  graphical  user  interface.  Screen-captures  from  the  program  are  shown  in  Figure  6.  The  package 
runs  on  an  ordinary  laptop  that  sends  reconstructed  seed  positions  to  Interplant®  system.  In  order  to  not  require 
a  new  FDA  approval,  we  maintain  the  integrity  of  the  FDA-approved  Interplant®  system  by  not  modifying  the 
commercial  software.  We  instead  use  a  text  file  to  export  the  3D  seed  locations. 


Dosimetry  Analysis  and  Implant  Optimization:  The  seed  locations  (in  template  coordinates)  are  ex¬ 
ported  to  the  Interplant®  system.  A  software  patch  added  to  the  Interplant®  removes  the  already  implanted 
seeds  from  the  original  plan,  thereby  producing  a  v  residual  implant  plan The  total  dose  is  calculated  by  com¬ 
bining  the  current  seed  locations  and  those  in  the  residual  implant  plan.  The  physician  at  this  stage,  modifies 
the  residual  plan  to  avoid  any  potential  ’hot  spots’  (prostate  tissue  receiving  significantly  more  than  prescribed 
radiation  dose),  and  most  importantly,  fills  in  any  observed  cold  spots  (under-dosed  prostate  tissue  receiving  less 
than  prescribed  radiation).  This  process  can  be  repeated  any  number  of  times  during  the  surgery,  achieving  a 
more  homogenous  seed  distribution. 


Clinical  Workflow:  The  procedure  flows  in  two  main  branches  as  shown  in  Figure  6  (b).  In  addition  to  the 
FDA  approved  TRUS-template  calibration,  the  pre-operative  phase  now  also  includes  the  C-arm  calibration. 
Intra-operatively,  the  procedure  proceeds  without  any  alterations.  At  any  point,  the  physician  can  decide  to 
incorporate  the  current  3D  seed  locations,  in  which  an  X-ray  reconstruction  is  carried  out.  The  TRUS  probe  is 
retracted  to  prevent  the  transducer  from  blocking  the  C-arm’s  field  of  view.  With  moving  the  C-arm  over  the 
target  area,  3-4  X-ray  images  are  collected.  Each  image  provides  a  view  of  the  prostate  (with  the  implanted 
seeds)  and  the  FTRAC  fiducial.  These  images  are  automatically  captured  by  the  computer  using  a  framegrabber. 
The  intra-operative  workflow  continues  with  extensive  image  analysis,  as  described  so  far.  Each  C-arm  image  is 
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Figure  6.  (a)  GUI  screen  captures  of  the  main  program,  offline  calibration,  seed  &;  fiducial  segmentation,  seed  matching 
and  validation  of  the  reconstruction  by  back-projection,  (b)  Workflow  for  intra-operative  dose  optimization. 


automaatically  corrected  for  distortion.  Next,  the  segmentation  algorithm  segments  the  motifs  of  the  FTRAC 
fiducial  and  recovers  the  relative  poses  of  the  C-arm  images,  followed  by  the  segmentation  of  all  visible  seeds  in 
the  images.  MARSHAL  then  matches  all  the  segmented  seeds  and  reconstructs  their  3D  locations,  while  also 
recovering  the  seeds  hidden  in  one  or  more  images.  Finally,  we  apply  the  predetermined  coordinate  transformation 
to  the  seed  cloud  to  obtain  the  position  of  the  seeds  in  template  coordinates  and  send  them  to  the  Interplant®, 
where  the  dose  coverage  relative  to  the  relevant  anatomy  is  analyzed  and  the  remainder  of  the  implant  plan 
optimized.  At  this  point,  the  clinical  workflow  reeverts  back  to  the  default  standard  of  care  protocol. 

3.  PHANTOM  EXPERIMENTS  AND  RESULTS 

We  have  extensively  tested  the  system  and  its  components  in  various  phantoms  and  in  a  Phase- 1  clinical  trial. 
To  do  so,  we  introduce  the  terms  absolute  and  relative  reconstruction  errors.  Using  X-ray  images,  the  seeds 
are  reconstructed  with  respect  to  the  FTRAC  frame.  In  experiments  where  the  ground  truth  location  of  the 
seeds  with  respect  to  the  FTRAC  is  known,  the  comparative  analysis  of  the  seed  locations  is  called  absolute 
accuracy.  Sometimes  (eg.  in  patients),  the  true  seed  locations  with  respect  to  the  FTRAC  are  not  available 
and  the  reconstruction  can  only  be  compared  to  the  seeds  extracted  from  post-operative  data  (using  a  rigid 
point-cloud  to  point-cloud  transform),  in  which  case  the  evaluation  is  called  relative  accuracy. 


Figure  7.  (a)  An  image  of  the  seed  phantom  attached  to  the  FTRAC  fiducial.  The  phantom  can  replicate  any  implant 
configuration,  using  the  twelve  5  mm  slabs  each  with  over  a  hundred  holes,  (b)  A  typical  X-ray  image  of  the  combination. 


Solid  Seed  Phantom:  An  acetol  (Delrin)  phantom  consisting  of  ten  slabs  (5mm  each)  was  fabricated  (Figure 
7).  This  phantom  provides  a  multitude  of  implants  with  sub- mm  ground  truth  accuracy.  The  fiducial  was  rigidly 
attached  to  the  phantom  in  a  known  way,  establishing  the  accurate  ground  truth  3D  location  of  each  seed. 
Realistic  prostate  implants  (1.56  seeds/cc,  40-100  seeds)  were  imaged  within  a  30°  cone  around  the  AP-axis. 
The  true  correspondence  was  manually  established  by  using  the  3D  locations,  known  from  the  precise  fabrication. 
Averaged  results  indicate  that  we  correctly  match  98.5%  &  99.8%  of  the  seeds  using  3  &  4  images  (100  &  75 
total  trials)  respectively.  The  mean  3D  absolute  reconstruction  accuracy  was  0.66  mm  (STD  0.29  mm),  while 
the  relative  accuracy  was  0.35  mm.  Furthermore,  using  4  images  yielded  only  one  poorly  mis-matched  seed  from 
the  75  datasets,  suggesting  the  use  of  4  images  for  better  clinical  guidance. 
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Figure  8.  An  annotated  image  of  the  (a)  experimental  setup  for  the  training  phantom  experiments;  (b)  overall  set  up  of 
the  full  operating  area  during  the  Phase-I  clincal  trials. 


Soft  Training  Phantoms:  As  annotated  in  Figure  8,  we  fully  seeded  three  standard  prostate  brachytherapy 
phantoms  with  realistic  implant  plans  (45,  49,  87  seeds).  Seed  locations  reconstructed  from  fluoro  using  realis¬ 
tic  (maximum  available  clinically)  image  separation  (about  15°)  were  compared  to  their  corresponding  ground 
truth  locations  segmented  manually  in  CT  (1mm  slice  thickness).  Additionally,  the  45  &  87-seed  phantoms  were 
rigidly  attached  to  the  FTRAC,  providing  the  absolute  ground  truth  (from  CT).  The  49-seed  phantom  was  used 
to  conduct  a  full  scale  practice-surgery,  in  which  case  the  3D  reconstruction  could  be  compared  only  to  the  seed 
cloud  from  post-op  CT  (without  FTRAC),  providing  just  relative  accuracy. 

The  performance  of  the  system  on  the  soft  tissue  training  phantoms  is  tabulated  in  Table  1.  The  absolute 
reconstruction  errors  for  the  45,  87-seed  phantoms  were  1.64  mm  &  0.95  mm  (STD  0.17  mm),  while  the  relative 
reconstruction  errors  for  the  45,  49,  87-seed  phantoms  were  0.22  mm,  0.29  mm,  0.20  mm  (STD  0.13  mm).  A 
mean  translation  shift  of  1.32  mm  was  observed  in  the  3D  reconstructions,  predominantly  due  to  the  limited 
C-arm  workspace  (solid-phantom  experiments  with  30°  motion  have  0.65  mm  accuracy).  It  was  observed  that  the 
shift  was  mostly  random  &  not  in  any  particular  direction.  In  fact,  it  was  noticed  that  if  different  combinations 
of  the  X-ray  images  were  used,  it  would  result  in  different  directions  for  the  overall  shift.  This  suggests  that  the 
shift  is  most  probably  due  to  the  small  amounts  of  error  (sub-mm,  sub-degree)  in  the  pose  estimation  part  of  the 
system.  Note  that  our  reconstruction  accuracy  is  better  than  the  CT  resolution  (as  evident  from  the  solid  seed 
phantom  experiments).  Moreover,  the  3D  error  is  larger  in  the  Z-direction,  which  corresponds  to  the  direction 


Number 
of  Seeds 

Error  Type 

Error  (mm) 

X  Y  Z 

3D  Error 
(mm) 

Translation 
Offset  (mm) 

15 

Absolute 

0.38 

0.94 

1.27 

1.64 

1.50 

87 

0.41 

0.40 

0.71 

0.95 

1.13 

Average 

040 

0.67 

0.99 

1.30 

1.32 

45 

Relative 

0.08 

0.14 

0.12 

0.22 

- 

49 

0.09 

0.11 

0.10 

0.29 

- 

87 

0.12 

0.18 

0.12 

0.20 

- 

Average 

0.10 

0.14 

0.11 

0.24 

- 

Table  1.  Tabulated  performance  of  the  system  on  soft  tissue  training  phantoms.  Note  the  difference  between  the  absolute 
and  relative  reconstruction  errors. 


of  the  1  mm  axial  slices  of  the  CT.  Nevertheless,  the  accuracy  is  sufficient  for  brachytherapy,  especially  since  a 
small  shift  does  not  hamper  the  ability  to  detect  the  presence  of  any  cold  spots. 


Patients:  A  total  of  18  batches  of  reconstructions  were  carried  out  on  6  patients  with  2  —  4  batches/patient, 
as  and  when  a  need  for  dose  evaluation  was  felt  by  the  clinician.  The  number  of  seeds  in  each  batch  varied  from 
22  —  84.  Since  the  seeds  migrate  significantly  by  the  time  a  post-operative  CT  is  taken,  there  is  no  easy  method 
for  knowing  the  true  3D  seed  locations  (ground  truth)  in  real  patients.  Hence,  for  each  reconstruction  4  —  6 
additional  X-ray  images  were  taken.  The  intra-operatively  reconstructed  3D  seed  locations  were  projected  on 
these  additional  images  and  compared  to  their  corresponding  segmented  2D  locations  (post-operatively) .  The 
results  from  a  total  of  99  such  projections  from  18  reconstructions  are  tabulated  in  Table  2. 


Patient 

Number 

Additional 

Error 

in  X-ray  Image  (mm) 

Number 

of  Seeds 

X-ray  Images 

Mean 

STD 

Max 

Min 

1 

22 

6 

2.32 

0.21 

2.75 

1.92 

44 

5 

2.24 

0.31 

2.86 

1.47 

65 

5 

1.19 

0.29 

1.95 

0.52 

66 

5 

1.01 

0.23 

1.70 

0.53 

2 

39 

5 

1.20 

0.21 

1.67 

0.81 

84 

4 

2.16 

0.60 

4.18 

0.35 

3 

33 

5 

1.19 

0.27 

1.88 

0.63 

67 

6 

1.60 

0.44 

2.76 

0.74 

70 

4 

1.47 

0.64 

3.22 

0.38 

4 

35 

5 

1.21 

0.21 

1.64 

0.75 

68 

6 

1.43 

0.42 

3.05 

0.41 

77 

5 

1.81 

0.29 

2.59 

1.15 

5 

24 

6 

2.30 

0.45 

3.09 

1.46 

48 

6 

2.19 

0.46 

3.51 

1.07 

53 

7 

2.17 

0.41 

3.40 

1.51 

6 

33 

6 

1.90 

0.32 

2.67 

1.34 

61 

7 

2.30 

0.34 

3.44 

1.49 

66 

6 

1.40 

0.29 

2.14 

0.76 

Average 

56 

5.5 

1.57 

0.34 

2.50 

0.83 

Table  2.  18  intra-operative  3D  X-ray  seed  reconstructions  with  22-84  seeds,  projected  back  on  additional  99  (unused) 
X-ray  images.  Note  that  the  maximum  error  is  never  remarkably  huge. 


The  results  indicate  a  2D  mean  error  of  1.57  mm  (STD  0.34  mm,  max  2.50  mm,  mm0.83  mm).  This 
indicates  a  sub-mm  accuracy  in  3D  reconstruction,  since  the  errors  get  magnified  under  a  perspective  transform 
when  projected  from  3D  to  a  2D  image.  Though  the  magnification  factor  varies  for  each  image  depending  on  the 
relative  depth  of  the  seeds  with  respect  to  the  C-arm  focal  length,  it  ranges  approximately  from  1.5  —  2.  This 
indicates  an  average  absolute  3D  reconstruction  accuracy  of  0.8  —  1.0  mm,  which  is  similar  to  the  results  obtained 
from  the  phantom  experiments.  Further  note  that  the  average  minimum  error  is  0.83  mm,  indicating  that  there 
is  a  small  consistent  shift/bias  in  the  reconstruction.  This  collaborates  completely  with  the  observations  that 
were  made  earlier.  In  only  one  case  (patient-2  84-seeds),  we  see  a  maximum  error  more  than  4  mm.  However, 
in  this  cases  we  also  observe  that  the  overall  deviation  is  higher  too,  indicating  that  these  datasets  might  have 
a  larger  3D  shift/bias.  In  any  case,  we  do  not  see  the  maximum  error  go  many  deviations  away  from  the  mean 
error.  A  2.5  mm  error  in  3D  is  expected  to  project  as  a  5  mm  error  at  least  in  some  of  the  X-ray  images,  which  we 
have  not  observed.  This  indicates  that  we  had  successful  intra-operative  reconstructions  in  the  Phase-I  patient 
trials. 


Registration  Accuracy:  To  measure  the  accuracy  of  the  fiducial-to-template  registration,  three  batches  of 
five  straight  needles  each  were  inserted  randomly  at  known  depths  (Z-axis)  into  known  template  holes  (X, 
Y-axis).  Their  reconstructed  tip  locations  with  respect  to  the  FTRAC  coordinate- frame  were  transformed  to 
the  template  coordinate-frame  using  our  rigid  registration  transform,  which  were  then  compared  to  their  true 
measured  locations  in  template  coordinates.  The  limited- angle  image-capture  protocol  was  kept  similar  to  that 
used  in  the  operating  room.  Both  absolute  and  relative  results  are  tabulated  in  Table  3.  The  average  absolute 
error  (reconstruction  together  with  registration)  was  1.03  mm  (STD  0.18  mm),  while  the  average  relative  error 
was  0.36  mm  (STD  0.21  mm),  with  an  average  translation  shift  of  0.97  mm.  Note  that  the  performance  of  the 
system  in  these  experiments  is  similar  to  the  performance  of  the  system  in  the  other  experiments  presented  so 
far,  strengthening  the  validity  of  our  conclusions. 


Trial 

Error 

Error  (mm) 

3D  Error 

Translation 

Number 

Type 

X 

Y 

Z 

(mm) 

Offset  (mm) 

1 

0.32 

0.70 

0.84 

1.16 

1.15 

2 

Absolute 

0.66 

0.29 

1.02 

1.27 

1.22 

3 

0.12 

0.28 

0.54 

0.65 

0.55 

1 

0.11 

0.17 

0.05 

0.22 

- 

2 

Relative 

0.10 

0.27 

0.35 

0.49 

- 

3 

0.10 

0.27 

0.19 

0.36 

- 

Table  3.  3  sets  of  5  needles  were  inserted  into  the  template  at  known  depths  and  reconstructed.  This  comparison  gives 
us  the  cumulative  3D  error  from  X-ray  reconstruction  with  respect  to  FTRAC  and  FTRAC-template  registration,  i.e.  3D 
reconstruction  error  with  respect  to  the  template. 


System  Accuracy:  To  measure  the  full  system  error,  5  needles  (tips)  were  inserted  into  a  prostate  brachyther- 
apy  training  phantom,  reconstructed  in  3D  and  exported  to  the  Interplant®  software.  The  needle  tips  were  also 
segmented  manually  using  TRUS  images.  The  sagittal  images  were  used  for  measuring  the  depth  (Z-axis)  of  the 
needles  and  the  transverse  images  were  used  to  measure  the  planar  positions  (X, Y-axis),  providing  ground  truth 
from  Ultrasound.  The  mean  absolute  error  for  the  5  needle  tips  was  4  mm  (STD  0.53  mm),  with  a  transla¬ 
tion  shift  of  3.94  mm.  In  comparison,  the  relative  accuracy  for  the  complete  system  was  0.83  mm  (STD  0.18  mm). 


The  shift  can  mainly  be  attributed  to  (i)  an  error  in  the  Template-TRUS  pre-calibration  done  as  part  of 
current  clinical  practice,  resulting  in  a  shift  (^  3  mm);  and  (ii)  a  random  bias  in  the  3D  reconstruction  due 
to  limited  image  separation  (^  1  mm).  Nevertheless,  we  removed  this  shift  in  the  clinical  cases  by  applying 
a  translation-offset  to  the  reconstructed  X-ray  seed  coordinates.  Owing  to  the  small  size  of  the  prostate,  the 


rotation  offset  was  always  found  to  be  minimal  and  could  be  ignored.  The  resultant  translation  offset  was 
intra-operatively  estimated  by  comparing  the  centroid  of  the  reconstructed  seeds  with  that  of  the  planned  seed 
locations,  and  by  aligning  the  two  together.  Note  that  the  centroid  is  a  first-order  statistic  and  robust  to 
any  spatially  symmetric  noise/displacement  model.  Though  a  heuristic,  it  provided  excellent  qualitative  results 
according  to  the  surgeon,  who  read  the  visual  cues  at  the  reconstructed  seed  locations  in  TRUS  images.  Based  on 
the  experiments  so  far  and  the  surgeon’s  feedback,  the  overall  accuracy  of  the  system  is  expected  to  be  1  —  2  mm 
during  clinical  use. 


Phase-I  Clinical  Trial:  We  have  treated  a  cohort  of  6  patients,  as  part  of  a  DoD  Phase-I  clinical  trial  to 
test  the  efficacy  and  safety  of  the  system  (the  Phase-II  trial  is  currently  open  for  enrollment).  Annotated  pho¬ 
tographs  of  the  clinical  setup  are  shown  in  Figure  5  (c)  &  8  (b).  Intra-operative  dosimetry  was  performed  (i) 
halfway  during  the  surgery;  (ii)  at  the  end  of  the  surgery;  and  (iii)  after  all  the  additional  seeds  were  inserted. 
The  current  protocol  adds  about  15  minutes  for  each  reconstruction,  including  the  capture  of  5  extra  X-ray 
images  for  research  validation,  image  processing,  3D  seed  reconstruction,  and  dosimetry  optimization.  In  regular 
everyday  clinical  practice,  we  anticipate  the  need  for  only  a  single  exit-dosimetry  reconstruction,  increasing  the 
operating  time  only  by  about  10  minutes.  In  all  the  patients  the  final  dosimetry  detected  cold  spots  (Figure  9). 
The  clinician  grew  quickly  to  trust  the  system  in  detecting  cold  spots,  and  instead  minimized  potential  hot  spots 
during  the  surgery.  The  medical  team  found  the  intra-operative  visualization  of  under-dosed  regions  valuable, 
inserting  an  additional  4.17  seeds  on  an  average  to  make  the  100%  prescribed  iso-dose  contour  cover  the  prostate. 
All  patients  were  released  from  the  operating  room  with  satisfactory  outcomes. 
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Figure  9.  The  system  is  able  to  detect  cold  spots.  The  seed  locations  (and  corresponding  100/150%  iso-dose  contours) 
as  assumed  by  the  planning  system  (top)  and  as  computed  by  the  proposed  system  (bottom),  discovering  2  cold  spots 
in  this  slice.  4  seeds  have  drifted  out  of  the  slice,  while  2  have  migrated  significantly  within. 


The  intra-operative  detection  and  visualization  of  edema  was  also  made  possible  by  the  proposed  system,  as 
shown  in  Figure  10.  Edema  is  the  swelling  of  the  prostate  tissue  due  to  the  repeated  stress  from  the  punctures 
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Figure  10.  The  system  can  visualize  intra-operative  edema,  as  seen  for  patient  3  (mean  4.6  mm,  STD  2.4  mm,  max 
12.3  mm).  The  ’planned’  (red)  versus  the  ’reconstructed’  (blue)  seed  positions  as  seen  in  the  template  view.  A  trend  of 
outward  dispersion  from  their  initial  locations  is  observed. 


created  by  needle  insertions  and  the  high  radioactivity  from  the  dropped  seeds.  It  is  the  single- most  important 
source  of  bottleneck  towards  accurate  dose  delivery,  arguably  leaving  behind  cold  spots.  The  system  is  unique, 
in  that  it  is  the  first  of  a  kind  that  can  intra-operatively  quantify  the  edema  related  seed  migration.  The  system 
computed  significant  intra-operative  edema  in  all  the  patients,  with  an  average  tissue  expansion  of  3.84  mm 
(STD  2.13  mm).  The  seeds  (and  hence  the  prostate)  showed  a  clear  tendency  for  outward  migration  from  their 
drop  positions  (with  maximums  up  to  16  mm).  In  all  the  patients,  towards  the  end  of  the  surgery,  it  was  found 
that  the  apex  of  the  prostate  (surgeon-end)  was  under-dosed. 


Comparison  Type 

Error  (mm) 

X  Y  Z 

Norm  (mm) 

Mean  STD  Max 

Pre-op  TRUS  to  Day-0  Post-op  CT  (6  patients) 

1.93 

2.30 

2.28 

4.39 

2.60 

14.25 

Pre-op  TRUS  to  Day-30  Post-op  CT  (6  patients) 

2.07 

2.19 

2.55 

4.56 

2.78 

17.47 

Day-0  Post-op  CT  to  Day-30  Post-op  CT  (6  patients) 

1.62 

2.00 

1.93 

3.78 

2.72 

17.31 

Table  4.  The  pre-operative  planned  3D  seed  coordinates  (TRUS)  were  compared  to  their  corresponding  post-operative 
locations  computed  from  CT  volumes.  Significant  seed  migration  due  to  edema  can  be  observed. 


A  further  comparison  of  the  3D  final  implant  reconstructed  using  the  X-ray  images  to  Day-0  CT  (2  mm 
slices)  indicated  an  increased  average  post-operative  edema  of  4.39  mm  (STD  2.60  mm ),  indicating  a  further 
post-operative  seed  migration  in  a  few  hours  after  the  patient  leaves  the  operating  room.  However,  post-operative 
seed  migration  is  an  inherent  limitation  of  brachytherapy,  the  best  solution  being  a  calculated  intra-operative  over¬ 
dosing  of  the  patient.  This  makes  the  detection  of  any  intra-operative  edema  even  more  critical.  Brachytherapists 
have  been  traditionally  accommodating  for  it  by  over-dosing  the  patient.  Though  effective,  it  involves  some  skilled 
qualitative  guesswork  on  behalf  of  the  surgeon,  potentially  limiting  cancer  control  in  difficult  cases.  Note  that,  in 
such  a  scenario,  a  sub-mm  accuracy  in  the  placement  of  the  seeds  inside  the  prostate  could  have  a  lesser  impact 
on  long-term  cancer  control. 


Patient 

Number 

Comparison 

Type 

Error  (mm) 

X  Y  Z 

Norm  (mm) 
Mean  STD  Max 

Additional 
Seeds  Added 

1 

0.26 

0.41 

0.45 

0.77 

1.09 

9.00 

1 

2 

1.91 

1.88 

3.18 

4.65 

2.20 

12.23 

2 

3 

Intra-op  X-ray  to 

2.50 

2.13 

2.28 

4.59 

2.36 

12.26 

3 

4 

Pre-op  TRUS 

1.59 

1.57 

2.68 

4.27 

2.53 

16.02 

9 

5 

1.61 

2.67 

3.05 

4.98 

2.83 

16.19 

5 

6 

1.47 

2.30 

1.88 

3.78 

1.76 

9.82 

5 

Average 

1.56 

1.83 

2.25 

3.84 

2.13 

12.59 

4.17 

1 

1.88 

2.71 

3.32 

5.42 

2.46 

11.14 

- 

2 

2.65 

3.48 

2.87 

6.16 

2.96 

14.32 

- 

3 

Intra-op  X-ray  to 

1.33 

1.39 

1.30 

2.67 

2.13 

14.72 

- 

4 

Day-0  Post-op  CT 

1.63 

2.76 

3.23 

5.45 

2.60 

14.50 

- 

5 

1.47 

1.49 

2.70 

3.77 

2.53 

13.79 

- 

6 

1.74 

1.22 

1.44 

3.00 

2.70 

14.45 

- 

Average 

1.78 

2.37 

2.47 

4.41 

2.56 

13.82 

- 

1 

2.64 

3.62 

2.57 

5.78 

2.49 

12.04 

- 

2 

3.45 

3.84 

2.86 

6.84 

3.32 

21.75 

- 

3 

Intra-op  X-ray  to 

2.04 

2.55 

2.13 

4.70 

1.97 

11.17 

- 

4 

Day-30  Post-op  CT 

1.70 

2.40 

1.87 

4.07 

2.14 

12.51 

- 

5 

2.45 

3.27 

2.83 

5.70 

3.10 

21.87 

- 

6 

0.97 

1.51 

1.67 

2.91 

2.42 

12.49 

- 

Average 

2.21 

2.87 

2.32 

5.00 

2.57 

15.31 

- 

Table  5.  The  intra-operatively  computed  seed  coordinates  were  compared  those  obtained  from  planned  locations  in  TRUS 
and  post-operative  segmentations  from  CT.  Additional  seeds  were  added  for  each  patient  to  close  any  detected  cold  spots. 


4.  CONCLUSION,  SHORTCOMINGS  AND  FUTURE  WORK 

A  system  for  intra-operative  brachytherapy  seed  monitoring  has  been  presented.  In  precision-machined  hard 
phantoms  with  40-100  seeds,  we  correctly  reconstructed  99.8%  seeds  with  a  mean  3D  accuracy  of  0.35  mm. 
In  soft  tissue  phantoms  with  45-87  seeds  &  clinically  realistic  15°  C-arm  motion,  we  correctly  reconstructed 
100%  seeds  with  an  accuracy  of  0.24  mm.  18  intra-operative  reconstructions  from  a  phase-I  clinical  study  also 
show  a  correct  3D  reconstruction  with  under  1  mm  error.  This  accuracy  is  more  than  sufficient  for  accurate 
intra-operative  monitoring  of  the  plan,  since  the  system  was  able  to  measure  the  extent  of  intra-operative  seed 
migration,  showing  that  it  could  be  as  high  as  16.19  mm  (mean  3.84  mm).  The  proposed  work  was  also  success¬ 
fully  used  in  a  Phase-I  DoD  clinical  trial,  showing  usefulness  and  great  potential. 

The  system  (a)  requires  no  significant  hardware;  (b)  does  not  alter  the  current  clinical  workflow;  (c)  can  be 
used  with  any  C-arm;  (d)  integrates  easily  with  any  pre-existing  brachytherapy  installation;  &  (e)  is  economically 
feasible  and  scalable.  There  is  some  added  radiation  to  the  patient  from  the  X-ray  images,  though  insignificant 
when  compared  to  that  from  the  radioactive  seeds.  Though  not  critical,  primary  shortcomings  include  (a)  15 
minute  additional  surgery  time;  &  (b)  a  small  translation  bias.  Research  is  currently  underway  to  remove 
these  limitations,  and  to  conduct  large  scale  clinical  studies  using  clinical  indicators.  Commercialization  is  also 
currently  in  progress.  Furthermore,  the  proposed  system  is  the  first  of  its  kind  that  can  intra-operatively  detect 
seed  migration  using  any  non-isocentric  C-arm,  achieving  a  significantly  more  homogeneous  distribution  and 
avoiding  radiation  hot/cold-spots.  All  the  above  put  together,  promises  to  lead  to  a  paradigm  shift  in  the 
standard  of  care  for  image-guided  prostate  brachytherapy,  considerably  improving  the  patient  quality  of  life. 
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